-- Logs begin at Wed 2022-08-24 16:33:09 UTC, end at Wed 2022-08-24 17:49:38 UTC. -- Aug 24 17:06:35.079209 dv-cinder-851640-4-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 24 17:06:35.086429 dv-cinder-851640-4-iscsi devstack@keystone.service[81717]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 24 17:06:35.088472 dv-cinder-851640-4-iscsi devstack@keystone.service[81717]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Aug 24 17:06:35 2022] *** Aug 24 17:06:35.088472 dv-cinder-851640-4-iscsi devstack@keystone.service[81717]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 24 17:06:35.088472 dv-cinder-851640-4-iscsi devstack@keystone.service[81717]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 24 17:06:35.088472 dv-cinder-851640-4-iscsi devstack@keystone.service[81717]: nodename: dv-cinder-851640-4-iscsi Aug 24 17:06:35.088472 dv-cinder-851640-4-iscsi devstack@keystone.service[81717]: machine: x86_64 Aug 24 17:06:35.088472 dv-cinder-851640-4-iscsi devstack@keystone.service[81717]: clock source: unix Aug 24 17:06:35.088472 dv-cinder-851640-4-iscsi devstack@keystone.service[81717]: pcre jit disabled Aug 24 17:06:35.088472 dv-cinder-851640-4-iscsi devstack@keystone.service[81717]: detected number of CPU cores: 4 Aug 24 17:06:35.088986 dv-cinder-851640-4-iscsi devstack@keystone.service[81717]: current working directory: / Aug 24 17:06:35.088986 dv-cinder-851640-4-iscsi devstack@keystone.service[81717]: detected binary path: /usr/bin/uwsgi-core Aug 24 17:06:35.088986 dv-cinder-851640-4-iscsi devstack@keystone.service[81717]: your processes number limit is 47680 Aug 24 17:06:35.088986 dv-cinder-851640-4-iscsi devstack@keystone.service[81717]: your memory page size is 4096 bytes Aug 24 17:06:35.088986 dv-cinder-851640-4-iscsi devstack@keystone.service[81717]: detected max file descriptor number: 2048 Aug 24 17:06:35.088986 dv-cinder-851640-4-iscsi devstack@keystone.service[81717]: lock engine: pthread robust mutexes Aug 24 17:06:35.088986 dv-cinder-851640-4-iscsi devstack@keystone.service[81717]: thunder lock: enabled Aug 24 17:06:35.088986 dv-cinder-851640-4-iscsi devstack@keystone.service[81717]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 24 17:06:35.088986 dv-cinder-851640-4-iscsi devstack@keystone.service[81717]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 24 17:06:35.129460 dv-cinder-851640-4-iscsi devstack@keystone.service[81717]: Python main interpreter initialized at 0x55e270d5c9c0 Aug 24 17:06:35.129460 dv-cinder-851640-4-iscsi devstack@keystone.service[81717]: python threads support enabled Aug 24 17:06:35.129460 dv-cinder-851640-4-iscsi devstack@keystone.service[81717]: your server socket listen backlog is limited to 100 connections Aug 24 17:06:35.129460 dv-cinder-851640-4-iscsi devstack@keystone.service[81717]: your mercy for graceful operations on workers is 90 seconds Aug 24 17:06:35.129834 dv-cinder-851640-4-iscsi devstack@keystone.service[81717]: mapped 403077 bytes (393 KB) for 2 cores Aug 24 17:06:35.129909 dv-cinder-851640-4-iscsi devstack@keystone.service[81717]: *** Operational MODE: preforking *** Aug 24 17:06:35.130190 dv-cinder-851640-4-iscsi devstack@keystone.service[81717]: *** uWSGI is running in multiple interpreter mode *** Aug 24 17:06:35.130190 dv-cinder-851640-4-iscsi devstack@keystone.service[81717]: spawned uWSGI master process (pid: 81717) Aug 24 17:06:35.130393 dv-cinder-851640-4-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 24 17:06:35.131529 dv-cinder-851640-4-iscsi devstack@keystone.service[81717]: spawned uWSGI worker 1 (pid: 81731, cores: 1) Aug 24 17:06:35.131529 dv-cinder-851640-4-iscsi devstack@keystone.service[81717]: spawned uWSGI worker 2 (pid: 81732, cores: 1) Aug 24 17:06:35.131529 dv-cinder-851640-4-iscsi devstack@keystone.service[81717]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 24 17:06:39.509494 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 17:06:39.509494 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: from cryptography.utils import int_from_bytes Aug 24 17:06:39.510374 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 17:06:39.510374 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: from cryptography.utils import int_from_bytes Aug 24 17:06:39.510782 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 17:06:39.510782 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: from cryptography.utils import int_from_bytes Aug 24 17:06:39.510934 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 17:06:39.510934 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: from cryptography.utils import int_from_bytes Aug 24 17:06:39.560556 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 24 17:06:39.560714 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 24 17:06:39.560802 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 24 17:06:39.560901 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 24 17:06:39.560978 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] command line args: [] {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 24 17:06:39.561056 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] command line args: [] {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 24 17:06:39.561182 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 24 17:06:39.561271 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 24 17:06:39.561351 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] ================================================================================ {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 24 17:06:39.561436 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] ================================================================================ {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 24 17:06:39.561631 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] admin_token = **** {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:06:39.561719 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] admin_token = **** {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:06:39.561880 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] config_dir = [] {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:06:39.561958 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] config_dir = [] {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:06:39.562075 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:06:39.562213 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:06:39.562297 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] config_source = [] {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:06:39.562371 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] config_source = [] {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:06:39.562449 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] debug = True {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:06:39.562546 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] debug = True {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:06:39.562632 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:06:39.562884 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:06:39.563013 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:06:39.568849 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:06:39.568975 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:06:39.569050 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:06:39.569111 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:06:39.569169 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:06:39.569232 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:06:39.569296 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:06:39.569367 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] list_limit = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:06:39.569436 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] list_limit = None {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:06:39.569504 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] log_config_append = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:06:39.569563 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] log_config_append = None {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:06:39.569620 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:06:39.569683 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:06:39.569747 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] log_dir = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:06:39.569816 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] log_dir = None {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:06:39.569875 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] log_file = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:06:39.569938 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] log_file = None {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:06:39.569938 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:06:39.570047 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:06:39.570047 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:06:39.570163 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:06:39.570163 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:06:39.570271 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:06:39.570271 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:06:39.570386 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:06:39.570456 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:06:39.570522 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:06:39.570586 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:06:39.570651 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:06:39.570715 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:06:39.570784 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:06:39.570848 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:06:39.570932 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:06:39.571019 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:06:39.571086 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:06:39.571151 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:06:39.571221 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:06:39.571285 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:06:39.571349 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:06:39.571413 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:06:39.571472 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:06:39.571530 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:06:39.571595 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:06:39.571654 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:06:39.571717 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:06:39.571780 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:06:39.571840 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:06:39.571904 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.18/identity {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:06:39.571971 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.18/identity {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:06:39.572031 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] publish_errors = False {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:06:39.572095 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] publish_errors = False {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:06:39.572095 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:06:39.572219 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:06:39.572301 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:06:39.572388 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:06:39.572459 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:06:39.572569 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:06:39.572631 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:06:39.572708 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:06:39.572768 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:06:39.572836 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:06:39.572901 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] standard_threads = False {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:06:39.572961 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] standard_threads = False {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:06:39.573018 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:06:39.573075 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:06:39.573152 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:06:39.573217 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:06:39.573282 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:06:39.573345 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:06:39.573408 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] use_journal = False {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:06:39.573466 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] use_journal = False {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:06:39.573523 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] use_json = False {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:06:39.573588 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] use_json = False {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:06:39.573647 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] use_stderr = False {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:06:39.573704 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] use_stderr = False {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:06:39.573761 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] use_syslog = False {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:06:39.573818 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] use_syslog = False {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:06:39.573875 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:06:39.573939 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 17:06:39.573998 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.574055 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.574112 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.574168 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.574225 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.574360 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.574422 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.574486 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.574545 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.574602 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.574664 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.574729 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.574793 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.574857 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.576022 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.576169 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.576302 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.576444 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.578213 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.578382 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.578519 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.578613 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.578677 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.578760 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.578828 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.578936 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.579027 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.579102 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.579173 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.579253 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.579327 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.579404 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.579477 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.579554 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.579627 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.579706 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.579784 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.579857 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.579927 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.580004 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.580076 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.580158 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.580236 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.580307 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.580378 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.580453 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.580524 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.580602 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.580674 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.580745 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.580822 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.580899 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.581030 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.581105 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.581176 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.581252 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.581329 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.581407 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.581487 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.581560 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.581635 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.581706 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.581706 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.581839 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.581927 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.582000 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.582075 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.582146 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.582216 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.582285 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.582368 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.582441 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.582511 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.582581 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.582651 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.582726 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.582804 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.582876 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.582965 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.583033 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.583107 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.583169 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.583236 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.583296 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.583358 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.583422 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.583482 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.583569 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.583642 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.583703 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.583761 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.583819 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.583878 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.583936 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.584001 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.584062 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.584120 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.584178 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.584236 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.584294 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.584361 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.584428 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.584488 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.584554 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.584622 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.584686 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.584746 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.584815 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.584880 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.584939 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.585010 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.585085 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.585157 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.585236 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.585321 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.585388 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.585458 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.585536 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.585612 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.585686 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.585758 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.585824 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.585895 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.585961 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.586031 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.586105 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.586179 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.586258 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.586379 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.586494 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.586617 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.586707 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.586851 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.586966 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.587093 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.587171 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.587368 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.587465 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.587606 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.587685 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.587856 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.587976 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.588101 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.588191 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.588340 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.588420 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.588574 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.588655 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.588809 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.588967 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.589044 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.589121 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.589283 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.589363 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.589517 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.589607 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.589759 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.589838 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.589995 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.590079 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.590231 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.590319 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.590466 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.590557 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.590700 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.590780 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.590955 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.591050 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.591220 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.591310 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.591461 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.591601 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.591681 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.591755 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.591924 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.592058 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.592165 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.592245 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.592400 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.592486 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.592634 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.592724 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.594667 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.594770 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.594770 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.595661 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.595740 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.595822 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.595822 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.595953 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.596025 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.596096 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.596167 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.596247 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.596247 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.596388 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.596460 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.596532 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.596611 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.596685 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.596756 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.596827 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.596898 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.596898 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.597034 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.597107 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.597107 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] auth.external = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.597661 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.597747 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.597747 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.597888 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.597963 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.598035 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] auth.external = None {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.598133 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] auth.password = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.598215 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.598290 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] auth.token = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.598290 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.598290 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.598484 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.598560 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.598640 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.598714 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.598813 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.598975 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] auth.password = None {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.599078 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.599296 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.599379 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] auth.token = None {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.599537 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] credential.caching = True {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.599790 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.599869 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.600044 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.600249 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.600356 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.600437 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.600512 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.600680 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.600824 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.600945 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.601057 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.601222 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.601378 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.601578 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.601730 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.601871 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.602007 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.602137 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] credential.caching = True {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.602256 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.602465 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.602609 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.602715 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.602853 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.602993 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.603221 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.603295 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.603460 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] federation.caching = True {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.603538 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.603699 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.603777 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.603938 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.604008 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.604186 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.604265 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.604420 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.604491 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.604670 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.604740 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.604907 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.604981 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.605168 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.605238 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.605409 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.605480 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.605669 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.605742 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.605909 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.605978 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.606159 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.606264 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.606429 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] identity.caching = True {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.606500 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] federation.caching = True {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.606680 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.606751 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.606950 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.607024 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.607190 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.607270 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.607423 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.607492 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.607673 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.607752 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.607906 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.607982 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.608146 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.608216 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.608401 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.608477 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.609206 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.609285 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.609436 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.609506 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.609669 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.609748 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.609902 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.609973 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] identity.caching = True {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.610148 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.610219 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.610400 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.610730 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.610879 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.610998 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.611217 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.611486 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.611602 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.611683 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.611875 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.611961 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.612135 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.612216 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.612386 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.612474 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.612617 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.612698 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.612848 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.612928 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.613095 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.613224 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.613335 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.613423 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.613581 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.613691 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.613836 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.613925 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.614084 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.614165 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.614353 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.614433 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.614593 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.614681 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.614831 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.614936 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.615084 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.615165 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.615319 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.620353 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.620456 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.620534 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.620608 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.620680 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.620752 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.620839 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.620914 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.620987 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.621058 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.621130 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.621202 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.621286 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.621362 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.621434 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.621505 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.621577 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.621648 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.621727 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.621801 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.621873 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.621945 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.621945 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.622083 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.622083 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.622221 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.622292 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.622364 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.622436 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.622507 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.622507 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.622645 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.622645 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.622773 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.622852 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.622942 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.623030 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.623105 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.623176 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.623247 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.623318 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.623389 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.623459 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] ldap.user = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.623520 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.623580 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.623638 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.623708 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.623777 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.623850 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.623921 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.623992 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.624064 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.624135 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.624214 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.624287 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.624358 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.624419 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.624488 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] ldap.user = None {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.624559 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.624893 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.624976 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.625055 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.625127 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.625198 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.625269 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.625347 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.625421 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.625493 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.625565 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.625636 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.625707 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.625786 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.625859 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.625937 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.626010 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.626082 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.626153 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.626232 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.626305 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.626377 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.626448 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.626519 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.626590 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.626669 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.626743 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.626815 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.626886 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.626981 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.627043 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.627125 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.627200 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.627272 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.627344 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.627415 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.627486 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.627565 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.627638 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.627714 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.627787 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.627857 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.627929 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.628007 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.628080 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.628151 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.628222 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.628294 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.628365 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.628443 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.628517 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.628588 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.628660 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.628739 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.628811 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.628957 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.629033 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.629109 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.629264 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] resource.caching = True {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.629347 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.629520 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.629612 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.629775 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.629855 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.630017 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.630096 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.630257 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.630343 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.630491 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.630571 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.630726 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.630806 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.631002 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.631124 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] resource.caching = True {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.631243 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.631321 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.631486 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.631566 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.631720 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.631808 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] role.caching = True {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.631983 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.632065 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] role.driver = sql {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.632192 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.632273 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.632447 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.632545 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.632669 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.632750 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.632904 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.632984 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.633143 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.633232 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.633379 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] role.caching = True {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.633459 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.633626 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] role.driver = sql {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.633706 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.633859 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.633940 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.634108 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.634186 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.634343 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.634423 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.634591 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.634671 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.634838 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.634938 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.635117 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.635191 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.635381 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.635462 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.635613 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.635803 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.635884 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.636049 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.636139 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.636304 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.636385 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.636547 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.636626 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.636794 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.636881 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.637039 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.637119 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.637273 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.637466 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.637546 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.637708 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.637790 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.637959 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.638040 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.638211 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.638291 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.638460 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.638541 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.638708 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.638788 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.638960 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.639063 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.639217 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.639298 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.639467 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.639547 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.639729 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.639868 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.639948 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.640021 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.640183 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.640265 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.640415 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.640504 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] token.caching = True {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.640657 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.640736 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.640909 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.641032 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.641155 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.641242 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.641381 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.641467 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.641614 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.641694 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.641844 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.641933 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.642080 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] token.caching = True {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.642161 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.642316 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.642396 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.642562 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.642651 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.642799 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.642878 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.643065 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.643149 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.643303 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.647165 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.647165 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.647348 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.647422 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.647493 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.647575 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.647727 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.647853 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.647935 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.648008 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.648079 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.648151 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.648232 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] database.connection = **** {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.648314 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.648398 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.648473 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.648544 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.648616 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.648687 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.648758 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.648837 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.648911 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.648982 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.649042 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.649112 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.649191 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] database.connection = **** {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.649272 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.649356 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.649430 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.649502 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.649573 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.649644 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.649715 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.649793 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.649867 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.649938 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.650009 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.650080 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.650151 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.650295 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.650371 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.650443 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.650514 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.650586 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.650657 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.650735 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.650798 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.650868 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.650964 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.651038 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.651109 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81732) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 24 17:06:39.651195 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.651195 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.651195 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.651195 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.651195 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.651195 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 17:06:39.651195 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81731) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 24 17:06:39.679606 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81732) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:06:39.679754 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81731) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:06:39.679838 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81732) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:06:39.679910 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81731) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:06:39.680049 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81732) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:06:39.680136 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81731) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:06:39.688707 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81731) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:06:39.688830 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81732) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:06:39.716702 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81732) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:06:39.716937 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81732) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:06:39.717222 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81731) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:06:39.717458 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81731) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:06:39.779008 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81732) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:06:39.779177 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81731) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:06:39.829056 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81732) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:06:39.829290 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81732) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:06:39.829384 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81731) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:06:39.829601 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81731) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:06:39.985642 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81732) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:06:39.985871 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81732) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:06:39.986048 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81731) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:06:39.986139 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81732) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:06:39.986284 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81731) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:06:39.986370 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81732) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:06:39.986503 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81732) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:06:39.986597 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81731) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:06:39.986721 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81731) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:06:39.986804 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81732) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:06:39.987033 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81731) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:06:39.987257 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81731) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:06:40.016105 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81732) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:06:40.016222 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81731) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:06:40.016340 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81732) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:06:40.016461 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81731) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:06:40.016573 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81732) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:06:40.016692 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81731) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:06:40.016785 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81732) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:06:40.016920 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81731) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:06:40.017005 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81732) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:06:40.017140 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81731) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:06:40.017224 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81732) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:06:40.017355 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81731) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:06:40.017440 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81732) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:06:40.017570 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81731) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:06:40.028861 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81732) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:06:40.036731 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81731) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:06:40.049281 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81732) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:06:40.049517 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81732) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:06:40.049653 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81731) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:06:40.049741 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81732) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:06:40.049887 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81731) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:06:40.049986 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81732) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:06:40.050111 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81731) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:06:40.050195 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81732) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:06:40.050328 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81731) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:06:40.050412 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81732) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:06:40.050542 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81731) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:06:40.050635 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81732) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:06:40.050753 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81731) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:06:40.051075 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81731) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 17:06:40.067796 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.068005 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 17:06:40.068262 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81731) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.068350 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.068473 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81731) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 17:06:40.068559 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 17:06:40.068799 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.068882 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81731) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.069005 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81731) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 17:06:40.069090 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.069283 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81731) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.069546 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81731) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.069628 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.069821 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.070057 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81731) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.070322 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81731) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.079242 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.079526 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.079800 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.079899 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81731) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.080078 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81731) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.080162 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.080355 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81731) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.080642 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81731) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.087060 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81732) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:06:40.087664 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81731) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:06:40.089965 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81732) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:06:40.090255 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.090535 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81731) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:06:40.090639 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.090822 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.090923 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81731) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.091131 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81731) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.091214 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.091396 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.091490 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81731) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.091671 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.091753 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81731) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.091935 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81731) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.092014 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.092209 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81731) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.092328 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.092546 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81731) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.092642 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.092901 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.092995 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81731) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.093216 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81731) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.093491 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81731) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.116002 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.116847 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81731) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.117467 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81732) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:06:40.117754 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.118307 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81731) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:06:40.118591 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81731) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.122442 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81732) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:06:40.122729 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.123070 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.123316 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81731) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:06:40.123601 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81731) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.123884 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81731) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.130195 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81732) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:06:40.130487 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.131137 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81731) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:06:40.131424 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81731) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.134589 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81732) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:06:40.134884 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.135197 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.135482 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.135565 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81731) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:06:40.135758 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.135840 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81731) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.136036 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.136126 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81731) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.136311 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.136393 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81731) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.136584 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.136672 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81731) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.136882 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81731) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.137154 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81731) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.137422 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81731) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.153835 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.154118 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.154563 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81731) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.154845 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81731) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.158033 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81732) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:06:40.158737 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81731) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:06:40.161291 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.161590 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.167545 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81731) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.167545 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81731) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.167684 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81732) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:06:40.167751 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81731) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:06:40.168957 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81732) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:06:40.169687 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81731) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:06:40.172752 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.173051 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.173339 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.173514 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81731) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.173610 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.173812 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81731) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.173894 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.174098 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81731) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.174180 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.174384 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81731) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.174662 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81731) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.174970 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81731) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.190480 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81732) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:06:40.190778 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.191076 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.191349 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.191495 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81731) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:06:40.191795 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81731) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.192067 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81731) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.192333 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81731) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.198207 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.199365 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81731) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.199782 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.200073 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.200948 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81731) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.201235 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81731) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.202736 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81732) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:06:40.203052 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.203359 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.203654 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.203923 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81731) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:06:40.204009 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.204212 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81731) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.204506 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81731) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.204803 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81731) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.205094 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81731) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.215242 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81732) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:06:40.215527 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.215814 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.216097 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.216351 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81731) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:06:40.216438 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.216639 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81731) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.216722 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.216918 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.217009 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81731) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.217196 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81731) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.217474 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81731) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.217818 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81731) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.218108 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81731) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.232304 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81732) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:06:40.234176 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81731) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:06:40.235895 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81732) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:06:40.237252 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81731) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:06:40.238759 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.240120 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81731) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.240224 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.241542 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81731) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.241701 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81732) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:06:40.241989 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.242292 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.243085 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81731) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:06:40.243373 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81731) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.243672 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81731) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.248781 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.251038 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81731) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.251134 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81732) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:06:40.252485 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81731) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:06:40.253989 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.254270 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.254562 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.254846 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.255510 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81731) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.255802 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81731) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.256088 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81731) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.256370 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81731) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.262823 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81732) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:06:40.263146 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.263435 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.264292 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81731) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:06:40.264582 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81731) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.264890 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81731) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.270330 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81732) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:06:40.270613 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.270919 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.271206 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.271501 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.271754 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81731) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 17:06:40.271840 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.272039 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81731) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.272122 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.272318 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.272408 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81731) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.272595 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81731) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.272677 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.272881 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.272962 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81731) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.273153 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81731) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.273243 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.273428 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.273510 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81731) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.273695 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81731) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.273776 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.273958 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81732) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.274041 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81731) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.274244 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81731) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.274526 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81731) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.274803 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81731) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.275166 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81731) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.275432 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81731) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 17:06:40.308790 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55e270d5c9c0 pid: 81732 (default app) Aug 24 17:06:40.310125 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55e270d5c9c0 pid: 81731 (default app) Aug 24 17:06:40.312518 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f5be30-4020-415e-8468-ae1b974dcb22 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:06:40.312827 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f5be30-4020-415e-8468-ae1b974dcb22 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:06:40.313084 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f5be30-4020-415e-8468-ae1b974dcb22 None None] PATH_INFO: `/v3` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:06:40.317249 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1/1] 10.222.0.18 () {50 vars in 791 bytes} [Wed Aug 24 17:06:40 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:06:47.491332 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c527268-571e-4670-96c9-d2a64bf06b52 None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:06:47.491926 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c527268-571e-4670-96c9-d2a64bf06b52 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:06:47.491926 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c527268-571e-4670-96c9-d2a64bf06b52 None None] PATH_INFO: `/` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:06:47.496079 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1/2] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:06:47 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 17:06:47.503096 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc56c496-13ca-4717-a843-a263bd4154ad None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:06:47.503497 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc56c496-13ca-4717-a843-a263bd4154ad None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:06:47.503677 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc56c496-13ca-4717-a843-a263bd4154ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:06:47.573740 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: INFO dbcounter [None req-fc56c496-13ca-4717-a843-a263bd4154ad None None] Registered counter for database keystone Aug 24 17:06:47.594552 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG oslo_db.sqlalchemy.engines [None req-fc56c496-13ca-4717-a843-a263bd4154ad None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81732) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 24 17:06:47.600859 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writer thread running {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 24 17:06:47.789298 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG passlib.handlers.bcrypt [None req-fc56c496-13ca-4717-a843-a263bd4154ad None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=81732) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 24 17:06:47.789640 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG passlib.handlers.bcrypt [None req-fc56c496-13ca-4717-a843-a263bd4154ad None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81732) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 24 17:06:47.844678 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-fc56c496-13ca-4717-a843-a263bd4154ad None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:06:47.846334 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-fc56c496-13ca-4717-a843-a263bd4154ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:06:47.973657 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 2/3] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:06:47 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 472 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Aug 24 17:06:48.446377 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c527268-571e-4670-96c9-d2a64bf06b52 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:06:48.514276 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: INFO dbcounter [None req-5c527268-571e-4670-96c9-d2a64bf06b52 None None] Registered counter for database keystone Aug 24 17:06:48.535956 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG oslo_db.sqlalchemy.engines [None req-5c527268-571e-4670-96c9-d2a64bf06b52 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81731) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 24 17:06:48.542032 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writer thread running {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 24 17:06:48.661355 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c527268-571e-4670-96c9-d2a64bf06b52 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:06:48.665657 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d3753b-8e31-4c96-a7b0-0806a3871434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:06:48.666296 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d3753b-8e31-4c96-a7b0-0806a3871434 admin admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:06:48.666584 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d3753b-8e31-4c96-a7b0-0806a3871434 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:06:48.666854 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d3753b-8e31-4c96-a7b0-0806a3871434 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:06:48.674754 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d3753b-8e31-4c96-a7b0-0806a3871434 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:06:48.727433 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: WARNING keystone.common.rbac_enforcer.enforcer [None req-35d3753b-8e31-4c96-a7b0-0806a3871434 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 24 17:06:48.727717 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d3753b-8e31-4c96-a7b0-0806a3871434 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:06:48.737301 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 2/4] 10.222.0.18 () {58 vars in 1324 bytes} [Wed Aug 24 17:06:48 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 17:06:48.743719 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc56c496-13ca-4717-a843-a263bd4154ad None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:06:48.759148 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc56c496-13ca-4717-a843-a263bd4154ad None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:06:48.763768 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9756eddf-16d6-4e86-bb56-4d5617eb09d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:06:48.765477 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9756eddf-16d6-4e86-bb56-4d5617eb09d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:06:48.765756 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9756eddf-16d6-4e86-bb56-4d5617eb09d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:06:48.766025 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9756eddf-16d6-4e86-bb56-4d5617eb09d8 admin admin] PATH_INFO: `/v3/projects/68a3eec6caa5424ea6bb39d5d6e46c2f` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:06:48.767613 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9756eddf-16d6-4e86-bb56-4d5617eb09d8 admin admin] RBAC: Authorizing `identity:get_project(project_id=68a3eec6caa5424ea6bb39d5d6e46c2f)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:06:48.819543 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: WARNING keystone.common.rbac_enforcer.enforcer [None req-9756eddf-16d6-4e86-bb56-4d5617eb09d8 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 24 17:06:48.819884 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9756eddf-16d6-4e86-bb56-4d5617eb09d8 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:06:48.824316 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 3/5] 10.222.0.18 () {56 vars in 1207 bytes} [Wed Aug 24 17:06:48 2022] GET /identity/v3/projects/68a3eec6caa5424ea6bb39d5d6e46c2f => generated 344 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:06:50.691024 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319fed3d-8f02-4a90-948c-5c3cfbdf076b None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:06:50.691531 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319fed3d-8f02-4a90-948c-5c3cfbdf076b None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:06:50.691615 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319fed3d-8f02-4a90-948c-5c3cfbdf076b None None] PATH_INFO: `/` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:06:50.695750 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 3/6] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:06:50 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 17:06:50.701060 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25616c42-5e7f-4a6b-830d-605809b279eb None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:06:50.701472 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25616c42-5e7f-4a6b-830d-605809b279eb None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:06:50.701612 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25616c42-5e7f-4a6b-830d-605809b279eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:06:50.722469 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-25616c42-5e7f-4a6b-830d-605809b279eb None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:06:50.724006 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-25616c42-5e7f-4a6b-830d-605809b279eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:06:50.731807 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 4/7] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:06:50 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 24 17:06:50.997689 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319fed3d-8f02-4a90-948c-5c3cfbdf076b None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:06:51.010623 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319fed3d-8f02-4a90-948c-5c3cfbdf076b None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:06:51.015390 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4463eb4-da14-4dd4-be97-9b00ef8f969a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:06:51.015990 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4463eb4-da14-4dd4-be97-9b00ef8f969a admin admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:06:51.016270 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4463eb4-da14-4dd4-be97-9b00ef8f969a admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:06:51.016545 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4463eb4-da14-4dd4-be97-9b00ef8f969a admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:06:51.024091 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4463eb4-da14-4dd4-be97-9b00ef8f969a admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:06:51.025849 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4463eb4-da14-4dd4-be97-9b00ef8f969a admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:06:51.034478 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 4/8] 10.222.0.18 () {58 vars in 1324 bytes} [Wed Aug 24 17:06:50 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 17:06:51.039732 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25616c42-5e7f-4a6b-830d-605809b279eb None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:06:51.048712 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25616c42-5e7f-4a6b-830d-605809b279eb None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:06:51.053039 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc37255-4611-4b93-a59f-fff25816aea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:06:51.053665 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc37255-4611-4b93-a59f-fff25816aea7 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:06:51.053944 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc37255-4611-4b93-a59f-fff25816aea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:06:51.054217 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc37255-4611-4b93-a59f-fff25816aea7 admin admin] PATH_INFO: `/v3/users/20b73f4d8b424258a870e87b04c9a532` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:06:51.055551 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc37255-4611-4b93-a59f-fff25816aea7 admin admin] RBAC: Authorizing `identity:get_user(user_id=20b73f4d8b424258a870e87b04c9a532)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:06:51.057375 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc37255-4611-4b93-a59f-fff25816aea7 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:06:51.058877 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 5/9] 10.222.0.18 () {56 vars in 1198 bytes} [Wed Aug 24 17:06:51 2022] GET /identity/v3/users/20b73f4d8b424258a870e87b04c9a532 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:06:53.048058 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7024bf5a-52d6-4f93-b0e5-f3b9b7b3d50f None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:06:53.048579 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7024bf5a-52d6-4f93-b0e5-f3b9b7b3d50f None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:06:53.048579 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7024bf5a-52d6-4f93-b0e5-f3b9b7b3d50f None None] PATH_INFO: `/` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:06:53.049732 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feac96dd-7cfa-4b87-84fa-08813d52cca1 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:06:53.050179 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 5/10] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:06:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:06:53.050267 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feac96dd-7cfa-4b87-84fa-08813d52cca1 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:06:53.050267 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feac96dd-7cfa-4b87-84fa-08813d52cca1 None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:06:53.051626 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 6/11] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:06:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 17:06:53.054798 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654551eb-57c2-4720-9ef4-ae57ebb3d5b7 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:06:53.055108 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654551eb-57c2-4720-9ef4-ae57ebb3d5b7 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:06:53.055384 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654551eb-57c2-4720-9ef4-ae57ebb3d5b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:06:53.056331 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4d5b93-feab-4490-8189-bff8379f0e47 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:06:53.056615 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4d5b93-feab-4490-8189-bff8379f0e47 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:06:53.056889 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4d5b93-feab-4490-8189-bff8379f0e47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:06:53.074444 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-cf4d5b93-feab-4490-8189-bff8379f0e47 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:06:53.075654 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-cf4d5b93-feab-4490-8189-bff8379f0e47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:06:53.098004 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG passlib.handlers.bcrypt [None req-654551eb-57c2-4720-9ef4-ae57ebb3d5b7 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=81731) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 24 17:06:53.098414 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG passlib.handlers.bcrypt [None req-654551eb-57c2-4720-9ef4-ae57ebb3d5b7 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81731) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 24 17:06:53.143957 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-654551eb-57c2-4720-9ef4-ae57ebb3d5b7 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:06:53.145520 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-654551eb-57c2-4720-9ef4-ae57ebb3d5b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:06:53.153144 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 7/12] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:06:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 17:06:53.241599 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 6/13] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:06:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:06:53.417897 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4d5b93-feab-4490-8189-bff8379f0e47 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:06:53.441299 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:06:53.508092 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654551eb-57c2-4720-9ef4-ae57ebb3d5b7 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:06:53.522256 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4d5b93-feab-4490-8189-bff8379f0e47 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:06:53.557737 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1412b92-be92-46c9-bb4d-57e7cebe174c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:06:53.558365 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1412b92-be92-46c9-bb4d-57e7cebe174c None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:06:53.558651 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1412b92-be92-46c9-bb4d-57e7cebe174c None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:06:53.558949 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1412b92-be92-46c9-bb4d-57e7cebe174c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:06:53.609980 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654551eb-57c2-4720-9ef4-ae57ebb3d5b7 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:06:53.612277 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:06:53.626089 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1412b92-be92-46c9-bb4d-57e7cebe174c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:06:53.627699 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1412b92-be92-46c9-bb4d-57e7cebe174c None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:06:53.647023 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99df851-5692-4aad-bcbf-bb9bd217fd04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:06:53.648435 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99df851-5692-4aad-bcbf-bb9bd217fd04 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:06:53.648773 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99df851-5692-4aad-bcbf-bb9bd217fd04 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:06:53.649189 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99df851-5692-4aad-bcbf-bb9bd217fd04 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:06:53.650746 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:06:53.661581 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99df851-5692-4aad-bcbf-bb9bd217fd04 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:06:53.670067 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-a99df851-5692-4aad-bcbf-bb9bd217fd04 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:06:53.670067 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:06:53.670067 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:06:53.670067 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:06:53.670067 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:06:53.670067 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:06:53.670067 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:06:53.670067 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:06:53.670067 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:06:53.670067 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:06:53.670067 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:06:53.670067 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:06:53.670067 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:06:53.670067 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:06:53.670067 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:06:53.670067 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:06:53.670998 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:06:53.670998 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:06:53.670998 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:06:53.670998 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:06:53.670998 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:06:53.670998 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:06:53.670998 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:06:53.670998 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:06:53.670998 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:06:53.670998 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:06:53.670998 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:06:53.670998 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application with Lock( Aug 24 17:06:53.670998 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:06:53.670998 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:06:53.670998 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:06:53.670998 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:06:53.672309 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:06:53.672309 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:06:53.672309 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:06:53.672309 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:06:53.672309 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:06:53.672309 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:06:53.672309 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:06:53.672309 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:06:53.672309 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:06:53.672309 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:06:53.672309 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:06:53.672309 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:06:53.672309 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 7/14] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Aug 24 17:06:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:06:53.677030 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99df851-5692-4aad-bcbf-bb9bd217fd04 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:06:53.728438 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 8/15] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 24 17:06:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 17:06:53.733988 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1412b92-be92-46c9-bb4d-57e7cebe174c None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:06:53.785349 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99df851-5692-4aad-bcbf-bb9bd217fd04 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:06:53.818713 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d0ec00-f5af-4e91-b2fc-f2e15cd60522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:06:53.819378 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d0ec00-f5af-4e91-b2fc-f2e15cd60522 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:06:53.819658 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d0ec00-f5af-4e91-b2fc-f2e15cd60522 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:06:53.819931 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d0ec00-f5af-4e91-b2fc-f2e15cd60522 None admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:06:53.820595 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d0ec00-f5af-4e91-b2fc-f2e15cd60522 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:06:53.820898 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d0ec00-f5af-4e91-b2fc-f2e15cd60522 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:06:53.822341 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d0ec00-f5af-4e91-b2fc-f2e15cd60522 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:06:53.833791 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1412b92-be92-46c9-bb4d-57e7cebe174c None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:06:53.833947 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 8/16] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 24 17:06:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:06:53.839697 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d0ec00-f5af-4e91-b2fc-f2e15cd60522 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:06:53.857869 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:06:53.867372 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:06:53.869737 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9740a5-395b-4aa7-a536-feeeffc0ba6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:06:53.870366 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9740a5-395b-4aa7-a536-feeeffc0ba6f None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:06:53.870647 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9740a5-395b-4aa7-a536-feeeffc0ba6f None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:06:53.870955 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9740a5-395b-4aa7-a536-feeeffc0ba6f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:06:53.881561 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9740a5-395b-4aa7-a536-feeeffc0ba6f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:06:53.883061 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9740a5-395b-4aa7-a536-feeeffc0ba6f None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:06:53.900758 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-2e9740a5-395b-4aa7-a536-feeeffc0ba6f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:06:53.900758 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:06:53.900758 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:06:53.900758 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:06:53.900758 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:06:53.900758 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:06:53.900758 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:06:53.900758 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:06:53.900758 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:06:53.900758 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:06:53.900758 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:06:53.900758 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:06:53.900758 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:06:53.900758 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:06:53.900758 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:06:53.900758 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:06:53.901640 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:06:53.901640 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:06:53.901640 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:06:53.901640 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:06:53.901640 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:06:53.901640 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:06:53.901640 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:06:53.901640 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 24 17:06:53.901640 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:06:53.901640 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:06:53.901640 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:06:53.901640 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:06:53.901640 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:06:53.901640 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:06:53.901640 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:06:53.901640 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:06:53.901640 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:06:53.902454 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:06:53.902454 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:06:53.902454 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:06:53.902454 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 9/17] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 24 17:06:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:06:53.908241 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9740a5-395b-4aa7-a536-feeeffc0ba6f None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:06:53.941268 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d0ec00-f5af-4e91-b2fc-f2e15cd60522 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:06:53.975804 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5dbb30-acd4-48a6-a9e9-e32c7b24fc43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:06:53.976445 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5dbb30-acd4-48a6-a9e9-e32c7b24fc43 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:06:53.976730 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5dbb30-acd4-48a6-a9e9-e32c7b24fc43 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:06:53.977007 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5dbb30-acd4-48a6-a9e9-e32c7b24fc43 None admin] PATH_INFO: `/v3/users/20b73f4d8b424258a870e87b04c9a532` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:06:53.978488 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5dbb30-acd4-48a6-a9e9-e32c7b24fc43 None admin] RBAC: Authorizing `identity:get_user(user_id=20b73f4d8b424258a870e87b04c9a532)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:06:53.979961 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5dbb30-acd4-48a6-a9e9-e32c7b24fc43 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:06:53.981543 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 9/18] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 24 17:06:53 2022] GET /identity/v3/users/20b73f4d8b424258a870e87b04c9a532 => generated 247 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:06:53.987565 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5dbb30-acd4-48a6-a9e9-e32c7b24fc43 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:06:54.005622 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9740a5-395b-4aa7-a536-feeeffc0ba6f None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:06:54.039033 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac0eb9b-8810-44fe-bbaf-9380ec028296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:06:54.039648 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac0eb9b-8810-44fe-bbaf-9380ec028296 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:06:54.039928 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac0eb9b-8810-44fe-bbaf-9380ec028296 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:06:54.040197 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac0eb9b-8810-44fe-bbaf-9380ec028296 None admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:06:54.040837 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac0eb9b-8810-44fe-bbaf-9380ec028296 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:06:54.041139 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac0eb9b-8810-44fe-bbaf-9380ec028296 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:06:54.042518 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac0eb9b-8810-44fe-bbaf-9380ec028296 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:06:54.059681 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 10/19] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 24 17:06:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:06:54.080834 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:06:54.088431 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5dbb30-acd4-48a6-a9e9-e32c7b24fc43 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:06:54.125691 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52892249-ae27-4e89-a648-2e6ca26b845d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:06:54.126343 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52892249-ae27-4e89-a648-2e6ca26b845d None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:06:54.126624 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52892249-ae27-4e89-a648-2e6ca26b845d None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:06:54.126894 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52892249-ae27-4e89-a648-2e6ca26b845d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:06:54.128080 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52892249-ae27-4e89-a648-2e6ca26b845d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:06:54.129542 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52892249-ae27-4e89-a648-2e6ca26b845d None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:06:54.131166 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 10/20] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 24 17:06:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:06:54.137298 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac0eb9b-8810-44fe-bbaf-9380ec028296 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:06:54.174606 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:06:54.242468 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac0eb9b-8810-44fe-bbaf-9380ec028296 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:06:54.278124 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515a92a5-3906-4fbe-ac2b-e51cb375628d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:06:54.278737 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515a92a5-3906-4fbe-ac2b-e51cb375628d None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:06:54.279065 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515a92a5-3906-4fbe-ac2b-e51cb375628d None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:06:54.279321 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515a92a5-3906-4fbe-ac2b-e51cb375628d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:06:54.280595 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515a92a5-3906-4fbe-ac2b-e51cb375628d None admin] RBAC: Adding query filter params (role.id=c5346bc54abf445f9900edf94264c850, scope.domain.id=default, user.id=20b73f4d8b424258a870e87b04c9a532) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:06:54.280911 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515a92a5-3906-4fbe-ac2b-e51cb375628d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:06:54.282358 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515a92a5-3906-4fbe-ac2b-e51cb375628d None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:06:54.289465 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 11/21] 10.222.0.18 () {56 vars in 1322 bytes} [Wed Aug 24 17:06:54 2022] GET /identity/v3/role_assignments?user.id=20b73f4d8b424258a870e87b04c9a532&scope.domain.id=default&role.id=c5346bc54abf445f9900edf94264c850 => generated 233 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:06:56.404090 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582818b1-bfaa-4195-81ba-72bf6484b6ed None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:06:56.405242 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582818b1-bfaa-4195-81ba-72bf6484b6ed None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:06:56.407527 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582818b1-bfaa-4195-81ba-72bf6484b6ed None None] PATH_INFO: `/` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:06:56.409437 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 11/22] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:06:56 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:06:56.414663 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aeb30e-ffbf-4581-9adc-8e213a018995 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:06:56.415561 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aeb30e-ffbf-4581-9adc-8e213a018995 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:06:56.416044 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aeb30e-ffbf-4581-9adc-8e213a018995 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:06:56.427860 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455e90d1-c5c0-4455-a0b4-62a408f3d45c None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:06:56.428444 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455e90d1-c5c0-4455-a0b4-62a408f3d45c None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:06:56.428925 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455e90d1-c5c0-4455-a0b4-62a408f3d45c None None] PATH_INFO: `/` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:06:56.430499 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 12/23] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:06:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 17:06:56.435687 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090d4ace-c195-43ee-9faf-bc5523f524ec None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:06:56.436835 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090d4ace-c195-43ee-9faf-bc5523f524ec None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:06:56.437343 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090d4ace-c195-43ee-9faf-bc5523f524ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:06:56.461833 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-79aeb30e-ffbf-4581-9adc-8e213a018995 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:06:56.463409 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-79aeb30e-ffbf-4581-9adc-8e213a018995 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:06:56.469648 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-090d4ace-c195-43ee-9faf-bc5523f524ec None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:06:56.471980 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-090d4ace-c195-43ee-9faf-bc5523f524ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:06:56.589063 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 13/24] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:06:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 17:06:56.607153 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 12/25] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:06:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:06:56.682007 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59aae8b-6358-4eac-80d2-e1928096cc20 None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:06:56.682292 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59aae8b-6358-4eac-80d2-e1928096cc20 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:06:56.682487 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59aae8b-6358-4eac-80d2-e1928096cc20 None None] PATH_INFO: `/` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:06:56.684247 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 14/26] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:06:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:06:56.689796 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a11e78-bb14-4bb5-8f23-cae2beae01e6 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:06:56.690086 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a11e78-bb14-4bb5-8f23-cae2beae01e6 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:06:56.690358 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a11e78-bb14-4bb5-8f23-cae2beae01e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:06:56.715013 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-a2a11e78-bb14-4bb5-8f23-cae2beae01e6 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:06:56.716247 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-a2a11e78-bb14-4bb5-8f23-cae2beae01e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:06:56.746912 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:06:56.809883 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 13/27] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:06:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 17:06:57.029297 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59aae8b-6358-4eac-80d2-e1928096cc20 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:06:57.084465 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f57905-a53b-4a48-9b1b-74151c160d54 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:06:57.084805 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f57905-a53b-4a48-9b1b-74151c160d54 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:06:57.085016 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f57905-a53b-4a48-9b1b-74151c160d54 None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:06:57.086554 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 14/28] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:06:57 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 17:06:57.099249 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46acab33-a5c9-4487-a021-d384fafa84ed None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:06:57.099530 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46acab33-a5c9-4487-a021-d384fafa84ed None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:06:57.099808 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46acab33-a5c9-4487-a021-d384fafa84ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:06:57.135075 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-46acab33-a5c9-4487-a021-d384fafa84ed None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:06:57.136711 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-46acab33-a5c9-4487-a021-d384fafa84ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:06:57.190693 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:06:57.209475 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59aae8b-6358-4eac-80d2-e1928096cc20 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:06:57.238052 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 15/29] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:06:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 17:06:57.238881 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46acab33-a5c9-4487-a021-d384fafa84ed None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:06:57.284146 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacd4055-5dff-40ca-b010-3b3412046e8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:06:57.285148 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacd4055-5dff-40ca-b010-3b3412046e8d None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:06:57.285673 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacd4055-5dff-40ca-b010-3b3412046e8d None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:06:57.286162 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacd4055-5dff-40ca-b010-3b3412046e8d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:06:57.309891 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacd4055-5dff-40ca-b010-3b3412046e8d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:06:57.315694 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-dacd4055-5dff-40ca-b010-3b3412046e8d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:06:57.315694 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:06:57.315694 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:06:57.315694 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:06:57.315694 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:06:57.315694 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:06:57.315694 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:06:57.315694 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:06:57.315694 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:06:57.315694 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:06:57.315694 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:06:57.315694 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:06:57.315694 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:06:57.315694 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:06:57.315694 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:06:57.315694 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:06:57.316563 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:06:57.316563 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:06:57.316563 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:06:57.316563 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:06:57.316563 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:06:57.316563 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:06:57.316563 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:06:57.316563 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:06:57.316563 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:06:57.316563 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:06:57.316563 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:06:57.316563 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application with Lock( Aug 24 17:06:57.316563 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:06:57.316563 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:06:57.316563 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:06:57.316563 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:06:57.317335 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:06:57.317335 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:06:57.317335 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:06:57.317335 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:06:57.317335 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:06:57.317335 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:06:57.317335 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:06:57.317335 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:06:57.317335 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:06:57.317335 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:06:57.317335 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:06:57.317335 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:06:57.324745 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 15/30] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Aug 24 17:06:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:06:57.325883 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacd4055-5dff-40ca-b010-3b3412046e8d None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:06:57.363505 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:06:57.378493 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46acab33-a5c9-4487-a021-d384fafa84ed None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:06:57.440151 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacd4055-5dff-40ca-b010-3b3412046e8d None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:06:57.451619 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b23aac-2c11-4471-bde1-d04585813c57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:06:57.456519 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b23aac-2c11-4471-bde1-d04585813c57 None admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:06:57.457148 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b23aac-2c11-4471-bde1-d04585813c57 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:06:57.457148 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b23aac-2c11-4471-bde1-d04585813c57 None admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:06:57.457759 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b23aac-2c11-4471-bde1-d04585813c57 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:06:57.465676 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b23aac-2c11-4471-bde1-d04585813c57 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:06:57.484498 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95b5a05-bc89-4c77-b953-9786680e3e95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:06:57.485874 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95b5a05-bc89-4c77-b953-9786680e3e95 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:06:57.485874 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95b5a05-bc89-4c77-b953-9786680e3e95 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:06:57.485874 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95b5a05-bc89-4c77-b953-9786680e3e95 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:06:57.506599 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95b5a05-bc89-4c77-b953-9786680e3e95 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:06:57.508340 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95b5a05-bc89-4c77-b953-9786680e3e95 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:06:57.525928 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-e95b5a05-bc89-4c77-b953-9786680e3e95 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:06:57.525928 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:06:57.525928 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:06:57.525928 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:06:57.525928 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:06:57.525928 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:06:57.525928 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:06:57.525928 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:06:57.525928 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:06:57.525928 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:06:57.525928 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:06:57.525928 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:06:57.525928 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:06:57.525928 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:06:57.525928 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:06:57.525928 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:06:57.526863 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:06:57.526863 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:06:57.526863 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:06:57.526863 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:06:57.526863 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:06:57.526863 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:06:57.526863 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:06:57.526863 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application with Lock( Aug 24 17:06:57.526863 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:06:57.526863 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:06:57.526863 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:06:57.526863 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:06:57.526863 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:06:57.526863 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:06:57.526863 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:06:57.526863 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:06:57.526863 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:06:57.527713 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:06:57.527713 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:06:57.527713 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:06:57.527713 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 16/31] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 24 17:06:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:06:57.528356 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95b5a05-bc89-4c77-b953-9786680e3e95 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:06:57.534668 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:06:57.653263 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95b5a05-bc89-4c77-b953-9786680e3e95 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:06:57.675346 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 16/32] 10.222.0.18 () {60 vars in 1131 bytes} [Wed Aug 24 17:06:57 2022] POST /identity/v3/roles => generated 219 bytes in 435 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:06:57.675346 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b23aac-2c11-4471-bde1-d04585813c57 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:06:57.700824 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befffe68-54c0-44d0-80ce-3ebc60728494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:06:57.702448 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befffe68-54c0-44d0-80ce-3ebc60728494 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:06:57.703014 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befffe68-54c0-44d0-80ce-3ebc60728494 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:06:57.703429 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befffe68-54c0-44d0-80ce-3ebc60728494 None admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:06:57.704661 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befffe68-54c0-44d0-80ce-3ebc60728494 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:06:57.705626 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befffe68-54c0-44d0-80ce-3ebc60728494 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:06:57.707847 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befffe68-54c0-44d0-80ce-3ebc60728494 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:06:57.722434 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 17/33] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 24 17:06:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:06:57.724421 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befffe68-54c0-44d0-80ce-3ebc60728494 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:06:57.812756 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:06:57.817191 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b23aac-2c11-4471-bde1-d04585813c57 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:06:57.867093 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befffe68-54c0-44d0-80ce-3ebc60728494 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:06:57.868206 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbfd676-bdf1-40f2-805b-d05566bf187f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:06:57.868836 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbfd676-bdf1-40f2-805b-d05566bf187f None admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:06:57.869120 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbfd676-bdf1-40f2-805b-d05566bf187f None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:06:57.869395 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbfd676-bdf1-40f2-805b-d05566bf187f None admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:06:57.870075 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbfd676-bdf1-40f2-805b-d05566bf187f None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:06:57.871944 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbfd676-bdf1-40f2-805b-d05566bf187f None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:06:57.907096 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd12d984-fd5e-4ce5-be09-3ec137d4b652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:06:57.908018 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd12d984-fd5e-4ce5-be09-3ec137d4b652 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:06:57.908480 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd12d984-fd5e-4ce5-be09-3ec137d4b652 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:06:57.908923 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd12d984-fd5e-4ce5-be09-3ec137d4b652 None admin] PATH_INFO: `/v3/domains` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:06:57.909753 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd12d984-fd5e-4ce5-be09-3ec137d4b652 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:06:57.910049 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd12d984-fd5e-4ce5-be09-3ec137d4b652 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:06:57.911442 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd12d984-fd5e-4ce5-be09-3ec137d4b652 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:06:57.929630 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 18/34] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 24 17:06:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:06:57.930732 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 17/35] 10.222.0.18 () {60 vars in 1131 bytes} [Wed Aug 24 17:06:57 2022] POST /identity/v3/roles => generated 225 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:06:57.931353 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90952cb0-4069-4129-8d98-40d25d6defd4 None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:06:57.931785 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90952cb0-4069-4129-8d98-40d25d6defd4 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:06:57.931912 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbfd676-bdf1-40f2-805b-d05566bf187f None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:06:57.932306 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90952cb0-4069-4129-8d98-40d25d6defd4 None None] PATH_INFO: `/` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:06:57.933886 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 19/36] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:06:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:06:57.935488 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90952cb0-4069-4129-8d98-40d25d6defd4 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:06:57.945218 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:06:58.075539 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90952cb0-4069-4129-8d98-40d25d6defd4 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:06:58.081468 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbfd676-bdf1-40f2-805b-d05566bf187f None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:06:58.110843 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:06:58.122622 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87db94ac-3208-444c-b0b4-d5058404522d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:06:58.123571 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87db94ac-3208-444c-b0b4-d5058404522d None admin] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:06:58.123995 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87db94ac-3208-444c-b0b4-d5058404522d None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:06:58.130624 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87db94ac-3208-444c-b0b4-d5058404522d None admin] PATH_INFO: `/v3/projects` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:06:58.130624 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87db94ac-3208-444c-b0b4-d5058404522d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:06:58.130624 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87db94ac-3208-444c-b0b4-d5058404522d None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:06:58.136042 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86e9696-703b-4e16-8f7f-c5c2065d00bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:06:58.136765 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86e9696-703b-4e16-8f7f-c5c2065d00bf None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:06:58.137023 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86e9696-703b-4e16-8f7f-c5c2065d00bf None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:06:58.137338 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86e9696-703b-4e16-8f7f-c5c2065d00bf None admin] PATH_INFO: `/v3/users/20b73f4d8b424258a870e87b04c9a532` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:06:58.138479 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86e9696-703b-4e16-8f7f-c5c2065d00bf None admin] RBAC: Authorizing `identity:get_user(user_id=20b73f4d8b424258a870e87b04c9a532)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:06:58.140519 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86e9696-703b-4e16-8f7f-c5c2065d00bf None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:06:58.142149 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 18/37] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 24 17:06:57 2022] GET /identity/v3/users/20b73f4d8b424258a870e87b04c9a532 => generated 247 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:06:58.149472 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9deb36-7827-4512-a9e7-a6f1eb5b99d6 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:06:58.150041 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9deb36-7827-4512-a9e7-a6f1eb5b99d6 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:06:58.150580 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9deb36-7827-4512-a9e7-a6f1eb5b99d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:06:58.171013 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-de9deb36-7827-4512-a9e7-a6f1eb5b99d6 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:06:58.172764 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-de9deb36-7827-4512-a9e7-a6f1eb5b99d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:06:58.239059 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 19/38] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:06:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:06:58.239888 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9deb36-7827-4512-a9e7-a6f1eb5b99d6 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:06:58.249786 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:06:58.344615 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 20/39] 10.222.0.18 () {60 vars in 1141 bytes} [Wed Aug 24 17:06:57 2022] POST /identity/v3/projects => generated 301 bytes in 401 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:06:58.344615 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c74716b-836f-41c4-b27c-8d943fb042ab None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:06:58.344615 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c74716b-836f-41c4-b27c-8d943fb042ab None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:06:58.344615 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c74716b-836f-41c4-b27c-8d943fb042ab None None] PATH_INFO: `/` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:06:58.344615 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 21/40] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:06:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:06:58.347029 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd8067a-c4a3-4a3a-9ebd-ede2b8620302 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:06:58.347400 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd8067a-c4a3-4a3a-9ebd-ede2b8620302 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:06:58.347759 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd8067a-c4a3-4a3a-9ebd-ede2b8620302 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:06:58.359016 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9deb36-7827-4512-a9e7-a6f1eb5b99d6 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:06:58.369176 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-9fd8067a-c4a3-4a3a-9ebd-ede2b8620302 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:06:58.370926 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-9fd8067a-c4a3-4a3a-9ebd-ede2b8620302 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:06:58.410040 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90206d51-e52d-4443-a6da-58749f09ff89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:06:58.411690 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90206d51-e52d-4443-a6da-58749f09ff89 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:06:58.412255 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90206d51-e52d-4443-a6da-58749f09ff89 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:06:58.413744 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90206d51-e52d-4443-a6da-58749f09ff89 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:06:58.415319 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90206d51-e52d-4443-a6da-58749f09ff89 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:06:58.417079 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90206d51-e52d-4443-a6da-58749f09ff89 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:06:58.421310 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 20/41] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 24 17:06:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:06:58.434008 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90206d51-e52d-4443-a6da-58749f09ff89 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:06:58.455886 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 22/42] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:06:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 17:06:58.514034 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd8067a-c4a3-4a3a-9ebd-ede2b8620302 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:06:58.520656 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:06:58.557039 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90206d51-e52d-4443-a6da-58749f09ff89 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:06:58.604555 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba81bf3-dcc5-4657-bf2d-6349eafdf1d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:06:58.605187 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba81bf3-dcc5-4657-bf2d-6349eafdf1d1 None admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:06:58.605470 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba81bf3-dcc5-4657-bf2d-6349eafdf1d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:06:58.605763 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba81bf3-dcc5-4657-bf2d-6349eafdf1d1 None admin] PATH_INFO: `/v3/domains/default/users/20b73f4d8b424258a870e87b04c9a532/roles/c5346bc54abf445f9900edf94264c850` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:06:58.608866 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba81bf3-dcc5-4657-bf2d-6349eafdf1d1 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=20b73f4d8b424258a870e87b04c9a532, role_id=c5346bc54abf445f9900edf94264c850)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:06:58.610383 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba81bf3-dcc5-4657-bf2d-6349eafdf1d1 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:06:58.641112 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd8067a-c4a3-4a3a-9ebd-ede2b8620302 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:06:58.650785 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:06:58.675177 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce6434b-a69e-4352-bf9a-fb3aa1b950c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:06:58.675784 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce6434b-a69e-4352-bf9a-fb3aa1b950c3 None admin] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:06:58.676065 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce6434b-a69e-4352-bf9a-fb3aa1b950c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:06:58.676342 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce6434b-a69e-4352-bf9a-fb3aa1b950c3 None admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:06:58.676996 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce6434b-a69e-4352-bf9a-fb3aa1b950c3 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:06:58.678500 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce6434b-a69e-4352-bf9a-fb3aa1b950c3 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:06:58.727300 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 21/43] 10.222.0.18 () {58 vars in 1361 bytes} [Wed Aug 24 17:06:58 2022] PUT /identity/v3/domains/default/users/20b73f4d8b424258a870e87b04c9a532/roles/c5346bc54abf445f9900edf94264c850 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:06:58.736092 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba81bf3-dcc5-4657-bf2d-6349eafdf1d1 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:06:58.760112 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 23/44] 10.222.0.18 () {60 vars in 1131 bytes} [Wed Aug 24 17:06:58 2022] POST /identity/v3/roles => generated 223 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:06:58.965038 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba81bf3-dcc5-4657-bf2d-6349eafdf1d1 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:06:59.001643 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089ebb34-9af0-4b83-8997-e31eff66d964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:06:59.002278 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089ebb34-9af0-4b83-8997-e31eff66d964 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:06:59.002558 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089ebb34-9af0-4b83-8997-e31eff66d964 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:06:59.002830 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089ebb34-9af0-4b83-8997-e31eff66d964 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:06:59.003991 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089ebb34-9af0-4b83-8997-e31eff66d964 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:06:59.005437 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089ebb34-9af0-4b83-8997-e31eff66d964 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:06:59.007016 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 22/45] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 24 17:06:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:06:59.012772 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce6434b-a69e-4352-bf9a-fb3aa1b950c3 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:06:59.111878 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce6434b-a69e-4352-bf9a-fb3aa1b950c3 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:06:59.144660 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320bc5a8-1f1c-4907-9c09-ef44de78a647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:06:59.145273 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320bc5a8-1f1c-4907-9c09-ef44de78a647 None admin] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:06:59.145557 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320bc5a8-1f1c-4907-9c09-ef44de78a647 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:06:59.145832 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320bc5a8-1f1c-4907-9c09-ef44de78a647 None admin] PATH_INFO: `/v3/projects` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:06:59.146483 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320bc5a8-1f1c-4907-9c09-ef44de78a647 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:06:59.148049 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320bc5a8-1f1c-4907-9c09-ef44de78a647 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:06:59.238261 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 24/46] 10.222.0.18 () {60 vars in 1141 bytes} [Wed Aug 24 17:06:59 2022] POST /identity/v3/projects => generated 312 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:07:00.633600 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2ce156-f409-4b30-8958-add3ee9d6827 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:00.634134 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2ce156-f409-4b30-8958-add3ee9d6827 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:00.634134 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2ce156-f409-4b30-8958-add3ee9d6827 None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:00.635519 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 23/47] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:07:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:07:00.641406 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1345269-7ae2-4a47-80c3-7c37a1fa0400 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:00.641798 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1345269-7ae2-4a47-80c3-7c37a1fa0400 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:00.641962 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1345269-7ae2-4a47-80c3-7c37a1fa0400 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:00.659540 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-e1345269-7ae2-4a47-80c3-7c37a1fa0400 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:07:00.660715 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-e1345269-7ae2-4a47-80c3-7c37a1fa0400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:07:00.681202 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:00.732675 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 25/48] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:07:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:07:00.994654 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2ce156-f409-4b30-8958-add3ee9d6827 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:00.997621 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:01.095363 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2ce156-f409-4b30-8958-add3ee9d6827 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:01.129156 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b91ab36-8405-4288-ae71-73a0b1047761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:01.129787 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b91ab36-8405-4288-ae71-73a0b1047761 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:01.130068 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b91ab36-8405-4288-ae71-73a0b1047761 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:01.130340 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b91ab36-8405-4288-ae71-73a0b1047761 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:01.141497 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b91ab36-8405-4288-ae71-73a0b1047761 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:07:01.144192 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b238937c-19bf-4921-b6e8-a47beeb24885 None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:01.144476 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b238937c-19bf-4921-b6e8-a47beeb24885 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:01.144755 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b238937c-19bf-4921-b6e8-a47beeb24885 None None] PATH_INFO: `/` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:01.146114 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 26/49] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:07:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 17:07:01.147974 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-8b91ab36-8405-4288-ae71-73a0b1047761 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:07:01.147974 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:07:01.147974 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:07:01.147974 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:07:01.147974 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:07:01.147974 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:07:01.147974 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:07:01.147974 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:07:01.147974 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:07:01.147974 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:07:01.147974 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:07:01.147974 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:07:01.147974 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:07:01.147974 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:07:01.147974 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:07:01.147974 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:07:01.148986 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:07:01.148986 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:07:01.148986 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:07:01.148986 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:07:01.148986 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:07:01.148986 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:07:01.148986 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:07:01.148986 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:07:01.148986 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:07:01.148986 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:07:01.148986 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:07:01.148986 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 24 17:07:01.148986 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:07:01.148986 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:07:01.148986 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:07:01.148986 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:07:01.149768 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:07:01.149768 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:07:01.149768 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:07:01.149768 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:07:01.149768 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:07:01.149768 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:07:01.149768 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:07:01.149768 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:07:01.149768 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:07:01.149768 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:07:01.149768 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:07:01.149768 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:07:01.149768 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 24/50] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Aug 24 17:07:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:07:01.151239 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b2a508-2255-4572-9909-faf97448723c None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:01.151524 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b2a508-2255-4572-9909-faf97448723c None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:01.151799 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b2a508-2255-4572-9909-faf97448723c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:01.154452 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b91ab36-8405-4288-ae71-73a0b1047761 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:01.180700 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-e2b2a508-2255-4572-9909-faf97448723c None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:07:01.181898 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-e2b2a508-2255-4572-9909-faf97448723c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:07:01.217059 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:01.249641 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 27/51] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:07:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:07:01.257345 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b91ab36-8405-4288-ae71-73a0b1047761 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:01.289630 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369e4a7f-c47f-472b-a18f-b926897539b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:01.290235 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369e4a7f-c47f-472b-a18f-b926897539b7 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:01.290518 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369e4a7f-c47f-472b-a18f-b926897539b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:01.290794 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369e4a7f-c47f-472b-a18f-b926897539b7 None admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:01.291487 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369e4a7f-c47f-472b-a18f-b926897539b7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:07:01.291792 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369e4a7f-c47f-472b-a18f-b926897539b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:01.293169 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369e4a7f-c47f-472b-a18f-b926897539b7 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:01.305191 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 25/52] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 24 17:07:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:07:01.310739 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b2a508-2255-4572-9909-faf97448723c None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:01.389106 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:01.408160 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b2a508-2255-4572-9909-faf97448723c None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:01.441951 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970e0040-045d-4de5-8304-3041b469ba04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:01.442591 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970e0040-045d-4de5-8304-3041b469ba04 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:01.442879 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970e0040-045d-4de5-8304-3041b469ba04 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:01.443190 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970e0040-045d-4de5-8304-3041b469ba04 None admin] PATH_INFO: `/v3/users/20b73f4d8b424258a870e87b04c9a532` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:01.444336 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970e0040-045d-4de5-8304-3041b469ba04 None admin] RBAC: Authorizing `identity:get_user(user_id=20b73f4d8b424258a870e87b04c9a532)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:01.445873 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970e0040-045d-4de5-8304-3041b469ba04 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:01.447483 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 28/53] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 24 17:07:01 2022] GET /identity/v3/users/20b73f4d8b424258a870e87b04c9a532 => generated 247 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:01.453340 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369e4a7f-c47f-472b-a18f-b926897539b7 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:01.514043 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970e0040-045d-4de5-8304-3041b469ba04 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:01.556874 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369e4a7f-c47f-472b-a18f-b926897539b7 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:01.582110 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:01.593385 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786bf45e-38dc-4f0f-9082-9ae4bedb2ab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:01.594201 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786bf45e-38dc-4f0f-9082-9ae4bedb2ab2 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:01.594487 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786bf45e-38dc-4f0f-9082-9ae4bedb2ab2 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:01.594759 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786bf45e-38dc-4f0f-9082-9ae4bedb2ab2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:01.596014 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786bf45e-38dc-4f0f-9082-9ae4bedb2ab2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:01.597998 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786bf45e-38dc-4f0f-9082-9ae4bedb2ab2 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:01.599091 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 26/54] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 24 17:07:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:01.604951 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786bf45e-38dc-4f0f-9082-9ae4bedb2ab2 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:01.615412 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970e0040-045d-4de5-8304-3041b469ba04 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:01.650191 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d81374-71c4-4dd0-9b0e-57731c70eaed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:01.650815 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d81374-71c4-4dd0-9b0e-57731c70eaed None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:01.651133 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d81374-71c4-4dd0-9b0e-57731c70eaed None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:01.651485 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d81374-71c4-4dd0-9b0e-57731c70eaed None admin] PATH_INFO: `/v3/domains/default` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:01.652605 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d81374-71c4-4dd0-9b0e-57731c70eaed None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:01.654066 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d81374-71c4-4dd0-9b0e-57731c70eaed None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:01.658916 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 29/55] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 24 17:07:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:07:01.661648 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d81374-71c4-4dd0-9b0e-57731c70eaed None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:01.679339 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:01.706403 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786bf45e-38dc-4f0f-9082-9ae4bedb2ab2 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:01.748883 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b38cc0e-dbca-4886-80d1-daee9e44add5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:01.749523 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b38cc0e-dbca-4886-80d1-daee9e44add5 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:01.749807 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b38cc0e-dbca-4886-80d1-daee9e44add5 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:01.750347 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b38cc0e-dbca-4886-80d1-daee9e44add5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:01.751274 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b38cc0e-dbca-4886-80d1-daee9e44add5 None admin] RBAC: Adding query filter params (role.id=c5346bc54abf445f9900edf94264c850, scope.domain.id=default, user.id=20b73f4d8b424258a870e87b04c9a532) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:07:01.751871 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b38cc0e-dbca-4886-80d1-daee9e44add5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:01.753811 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b38cc0e-dbca-4886-80d1-daee9e44add5 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:01.761487 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 27/56] 10.222.0.18 () {56 vars in 1322 bytes} [Wed Aug 24 17:07:01 2022] GET /identity/v3/role_assignments?user.id=20b73f4d8b424258a870e87b04c9a532&scope.domain.id=default&role.id=c5346bc54abf445f9900edf94264c850 => generated 530 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:07:01.880831 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d81374-71c4-4dd0-9b0e-57731c70eaed None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:01.916151 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c27bc94-bedb-4d7b-90e8-b03cf623bc3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:01.917002 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c27bc94-bedb-4d7b-90e8-b03cf623bc3a None admin] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:01.917284 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c27bc94-bedb-4d7b-90e8-b03cf623bc3a None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:01.917559 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c27bc94-bedb-4d7b-90e8-b03cf623bc3a None admin] PATH_INFO: `/v3/projects` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:01.918203 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c27bc94-bedb-4d7b-90e8-b03cf623bc3a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:01.919682 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c27bc94-bedb-4d7b-90e8-b03cf623bc3a None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:02.002973 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 30/57] 10.222.0.18 () {60 vars in 1140 bytes} [Wed Aug 24 17:07:01 2022] POST /identity/v3/projects => generated 298 bytes in 342 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:07:03.929638 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c589c96-7b34-420f-98ed-51ad9b73a318 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:03.930177 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c589c96-7b34-420f-98ed-51ad9b73a318 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:03.930177 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c589c96-7b34-420f-98ed-51ad9b73a318 None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:03.931555 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 28/58] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:07:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:07:03.936677 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8e8328-7201-481a-97be-f9b1bbe45e7f None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:03.937044 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8e8328-7201-481a-97be-f9b1bbe45e7f None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:03.937239 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8e8328-7201-481a-97be-f9b1bbe45e7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:03.955717 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-ef8e8328-7201-481a-97be-f9b1bbe45e7f None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:07:03.956930 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-ef8e8328-7201-481a-97be-f9b1bbe45e7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:07:04.016825 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:04.021948 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 31/59] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:07:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:07:04.288951 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c589c96-7b34-420f-98ed-51ad9b73a318 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:04.362914 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:04.409281 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c589c96-7b34-420f-98ed-51ad9b73a318 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:04.443478 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7ddf2a-7b63-4b6e-b021-bd77d9a657bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:04.444103 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7ddf2a-7b63-4b6e-b021-bd77d9a657bc None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:04.444386 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7ddf2a-7b63-4b6e-b021-bd77d9a657bc None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:04.444658 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7ddf2a-7b63-4b6e-b021-bd77d9a657bc None admin] PATH_INFO: `/v3/domains/default` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:04.445760 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7ddf2a-7b63-4b6e-b021-bd77d9a657bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:04.447219 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7ddf2a-7b63-4b6e-b021-bd77d9a657bc None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:04.448777 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 29/60] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 24 17:07:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:04.455710 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8e8328-7201-481a-97be-f9b1bbe45e7f None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:04.552186 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8e8328-7201-481a-97be-f9b1bbe45e7f None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:04.586234 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bf2c02-5483-4a2c-b3e2-4d86943f55e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:04.586859 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bf2c02-5483-4a2c-b3e2-4d86943f55e7 None admin] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:04.587181 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bf2c02-5483-4a2c-b3e2-4d86943f55e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:04.587457 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bf2c02-5483-4a2c-b3e2-4d86943f55e7 None admin] PATH_INFO: `/v3/users` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:04.588115 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bf2c02-5483-4a2c-b3e2-4d86943f55e7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:04.589557 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bf2c02-5483-4a2c-b3e2-4d86943f55e7 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:04.681137 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 32/61] 10.222.0.18 () {60 vars in 1132 bytes} [Wed Aug 24 17:07:04 2022] POST /identity/v3/users => generated 275 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:07:07.630163 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475b1c91-dbf7-4c83-91d6-184ba74b75a0 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:07.631137 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475b1c91-dbf7-4c83-91d6-184ba74b75a0 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:07.631624 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475b1c91-dbf7-4c83-91d6-184ba74b75a0 None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:07.633185 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 30/62] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:07:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:07:07.638881 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5257aa-39db-475f-833d-a99af1bc8fe3 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:07.640288 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5257aa-39db-475f-833d-a99af1bc8fe3 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:07.640768 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5257aa-39db-475f-833d-a99af1bc8fe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:07.662273 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-4a5257aa-39db-475f-833d-a99af1bc8fe3 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:07:07.663902 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-4a5257aa-39db-475f-833d-a99af1bc8fe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:07:07.708476 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addbf558-92bb-4b18-bcd8-2cd1c50f91f0 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:07.709103 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addbf558-92bb-4b18-bcd8-2cd1c50f91f0 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:07.709569 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addbf558-92bb-4b18-bcd8-2cd1c50f91f0 None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:07.711404 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 31/63] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:07:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:07:07.716785 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0911a62a-cc47-419b-a6b3-50cd86a89d7e None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:07.717998 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0911a62a-cc47-419b-a6b3-50cd86a89d7e None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:07.718508 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0911a62a-cc47-419b-a6b3-50cd86a89d7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:07.727273 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:07.746788 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 33/64] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:07:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:07:07.754937 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-0911a62a-cc47-419b-a6b3-50cd86a89d7e None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:07:07.756148 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-0911a62a-cc47-419b-a6b3-50cd86a89d7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:07:07.856131 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 32/65] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:07:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:07:08.096433 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5257aa-39db-475f-833d-a99af1bc8fe3 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:08.228540 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7863aac3-7e38-47c0-8806-efe153d6bfda None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:08.229196 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7863aac3-7e38-47c0-8806-efe153d6bfda None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:08.229697 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7863aac3-7e38-47c0-8806-efe153d6bfda None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:08.231450 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 33/66] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:07:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:07:08.231567 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5257aa-39db-475f-833d-a99af1bc8fe3 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:08.232561 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7863aac3-7e38-47c0-8806-efe153d6bfda None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:08.280916 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dad0804-6b7e-4678-bd03-578f244c762d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:08.281934 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dad0804-6b7e-4678-bd03-578f244c762d None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:08.282237 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dad0804-6b7e-4678-bd03-578f244c762d None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:08.282499 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dad0804-6b7e-4678-bd03-578f244c762d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:08.294566 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:08.295498 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dad0804-6b7e-4678-bd03-578f244c762d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:07:08.300570 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-8dad0804-6b7e-4678-bd03-578f244c762d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:07:08.300570 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:07:08.300570 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:07:08.300570 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:07:08.300570 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:07:08.300570 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:07:08.300570 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:07:08.300570 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:07:08.300570 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:07:08.300570 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:07:08.300570 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:07:08.300570 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:07:08.300570 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:07:08.300570 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:07:08.300570 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:07:08.300570 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:07:08.302262 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:07:08.302262 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:07:08.302262 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:07:08.302262 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:07:08.302262 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:07:08.302262 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:07:08.302262 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:07:08.302262 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:07:08.302262 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:07:08.302262 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:07:08.302262 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:07:08.302262 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application with Lock( Aug 24 17:07:08.302262 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:07:08.302262 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:07:08.302262 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:07:08.302262 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:07:08.309789 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:07:08.309789 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:07:08.309789 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:07:08.309789 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:07:08.309789 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:07:08.309789 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:07:08.309789 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:07:08.309789 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:07:08.309789 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:07:08.309789 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:07:08.309789 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:07:08.309789 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:07:08.309789 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 34/67] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 24 17:07:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:07:08.309789 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8879fb6-e0f7-4d4b-aea6-7d00dd725669 None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:08.310473 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8879fb6-e0f7-4d4b-aea6-7d00dd725669 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:08.310473 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8879fb6-e0f7-4d4b-aea6-7d00dd725669 None None] PATH_INFO: `/` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:08.310473 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 35/68] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:07:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:07:08.310473 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee57c50-1e3c-43b6-b714-5ebca6ae154e None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:08.310473 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee57c50-1e3c-43b6-b714-5ebca6ae154e None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:08.310473 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee57c50-1e3c-43b6-b714-5ebca6ae154e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:08.332706 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-5ee57c50-1e3c-43b6-b714-5ebca6ae154e None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:07:08.334037 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-5ee57c50-1e3c-43b6-b714-5ebca6ae154e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:07:08.353756 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7863aac3-7e38-47c0-8806-efe153d6bfda None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:08.375840 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:08.388692 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd92be6-02bb-4ff4-ab39-33f68a6d3d9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:08.389885 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd92be6-02bb-4ff4-ab39-33f68a6d3d9a None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:08.390166 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd92be6-02bb-4ff4-ab39-33f68a6d3d9a None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:08.390438 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd92be6-02bb-4ff4-ab39-33f68a6d3d9a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:08.401140 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd92be6-02bb-4ff4-ab39-33f68a6d3d9a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:07:08.405137 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 36/69] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:07:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:07:08.405934 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-bfd92be6-02bb-4ff4-ab39-33f68a6d3d9a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:07:08.405934 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:07:08.405934 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:07:08.405934 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:07:08.405934 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:07:08.405934 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:07:08.405934 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:07:08.405934 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:07:08.405934 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:07:08.405934 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:07:08.405934 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:07:08.405934 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:07:08.405934 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:07:08.405934 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:07:08.405934 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:07:08.405934 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:07:08.406786 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:07:08.406786 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:07:08.406786 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:07:08.406786 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:07:08.406786 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:07:08.406786 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:07:08.406786 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:07:08.406786 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:07:08.406786 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:07:08.406786 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:07:08.406786 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:07:08.406786 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 24 17:07:08.406786 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:07:08.406786 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:07:08.406786 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:07:08.406786 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:07:08.407898 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:07:08.407898 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:07:08.407898 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:07:08.407898 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:07:08.407898 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:07:08.407898 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:07:08.407898 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:07:08.407898 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:07:08.407898 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:07:08.407898 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:07:08.407898 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:07:08.407898 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:07:08.407898 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 34/70] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Aug 24 17:07:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:07:08.408833 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec5ddf1-5d86-46aa-94c6-dbb793ba739f None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:08.408833 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec5ddf1-5d86-46aa-94c6-dbb793ba739f None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:08.409101 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec5ddf1-5d86-46aa-94c6-dbb793ba739f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:08.412980 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee57c50-1e3c-43b6-b714-5ebca6ae154e None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:08.427757 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-8ec5ddf1-5d86-46aa-94c6-dbb793ba739f None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:07:08.429084 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-8ec5ddf1-5d86-46aa-94c6-dbb793ba739f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:07:08.503127 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 35/71] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:07:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:07:08.504672 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec5ddf1-5d86-46aa-94c6-dbb793ba739f None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:08.522117 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee57c50-1e3c-43b6-b714-5ebca6ae154e None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:08.560166 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:08.584030 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a800358f-d34b-42d8-9196-ff62e2dbb8ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:08.585179 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a800358f-d34b-42d8-9196-ff62e2dbb8ec None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:08.585568 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a800358f-d34b-42d8-9196-ff62e2dbb8ec None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:08.585916 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a800358f-d34b-42d8-9196-ff62e2dbb8ec None admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:08.587067 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a800358f-d34b-42d8-9196-ff62e2dbb8ec None admin] RBAC: Adding query filter params (name=member) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:07:08.587442 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a800358f-d34b-42d8-9196-ff62e2dbb8ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:08.588931 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a800358f-d34b-42d8-9196-ff62e2dbb8ec None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:08.607329 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 37/72] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Aug 24 17:07:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:08.609189 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d5ae1f-4bfe-493f-a905-88dc77e359e4 None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:08.609583 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d5ae1f-4bfe-493f-a905-88dc77e359e4 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:08.609915 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d5ae1f-4bfe-493f-a905-88dc77e359e4 None None] PATH_INFO: `/` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:08.615022 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 38/73] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:07:08 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:07:08.616039 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d5ae1f-4bfe-493f-a905-88dc77e359e4 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:08.648616 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec5ddf1-5d86-46aa-94c6-dbb793ba739f None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:08.671448 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:08.689813 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9fc8e1-211b-48be-8949-495ed120e932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:08.693442 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9fc8e1-211b-48be-8949-495ed120e932 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:08.693442 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9fc8e1-211b-48be-8949-495ed120e932 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:08.693442 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9fc8e1-211b-48be-8949-495ed120e932 None admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:08.693442 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9fc8e1-211b-48be-8949-495ed120e932 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:07:08.693442 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9fc8e1-211b-48be-8949-495ed120e932 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:08.694152 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9fc8e1-211b-48be-8949-495ed120e932 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:08.702668 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 36/74] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 24 17:07:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:08.704058 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3f7f15-663d-464f-9c17-63586a17ec7a None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:08.704350 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3f7f15-663d-464f-9c17-63586a17ec7a None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:08.704631 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3f7f15-663d-464f-9c17-63586a17ec7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:08.722503 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-9d3f7f15-663d-464f-9c17-63586a17ec7a None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:07:08.723736 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-9d3f7f15-663d-464f-9c17-63586a17ec7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:07:08.757247 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d5ae1f-4bfe-493f-a905-88dc77e359e4 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:08.791575 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b4b4ae-acd4-4f6f-bcc7-15ab3c7322b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:08.792270 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b4b4ae-acd4-4f6f-bcc7-15ab3c7322b7 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:08.792494 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b4b4ae-acd4-4f6f-bcc7-15ab3c7322b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:08.792763 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b4b4ae-acd4-4f6f-bcc7-15ab3c7322b7 None admin] PATH_INFO: `/v3/users/bd42523e33c34e618cc8f1f75df2b677` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:08.800062 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 37/75] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:07:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:07:08.800908 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3f7f15-663d-464f-9c17-63586a17ec7a None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:08.807991 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b4b4ae-acd4-4f6f-bcc7-15ab3c7322b7 None admin] RBAC: Authorizing `identity:get_user(user_id=bd42523e33c34e618cc8f1f75df2b677)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:08.809420 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b4b4ae-acd4-4f6f-bcc7-15ab3c7322b7 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:08.815627 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 39/76] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 24 17:07:08 2022] GET /identity/v3/users/bd42523e33c34e618cc8f1f75df2b677 => generated 275 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:08.815627 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b4b4ae-acd4-4f6f-bcc7-15ab3c7322b7 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:08.853083 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:08.908011 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:08.914683 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b4b4ae-acd4-4f6f-bcc7-15ab3c7322b7 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:08.929203 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3f7f15-663d-464f-9c17-63586a17ec7a None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:08.955395 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e588cda-05b1-48d3-946b-68c7200cabcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:08.956463 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e588cda-05b1-48d3-946b-68c7200cabcd None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:08.956750 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e588cda-05b1-48d3-946b-68c7200cabcd None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:08.957033 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e588cda-05b1-48d3-946b-68c7200cabcd None admin] PATH_INFO: `/v3/users/20b73f4d8b424258a870e87b04c9a532` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:08.958176 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e588cda-05b1-48d3-946b-68c7200cabcd None admin] RBAC: Authorizing `identity:get_user(user_id=20b73f4d8b424258a870e87b04c9a532)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:08.959727 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e588cda-05b1-48d3-946b-68c7200cabcd None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:08.965701 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 40/77] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 24 17:07:08 2022] GET /identity/v3/users/20b73f4d8b424258a870e87b04c9a532 => generated 247 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:08.965701 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e588cda-05b1-48d3-946b-68c7200cabcd None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:08.971747 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cc4470-03dc-4e02-ad49-039b8e1183bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:08.972449 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cc4470-03dc-4e02-ad49-039b8e1183bb None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:08.972737 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cc4470-03dc-4e02-ad49-039b8e1183bb None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:08.973017 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cc4470-03dc-4e02-ad49-039b8e1183bb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:08.985120 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cc4470-03dc-4e02-ad49-039b8e1183bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:07:08.989826 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-a9cc4470-03dc-4e02-ad49-039b8e1183bb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 17:07:08.989826 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:07:08.989826 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:07:08.989826 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:07:08.989826 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:07:08.989826 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:07:08.989826 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:07:08.989826 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:07:08.989826 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:07:08.989826 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:07:08.989826 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:07:08.989826 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:07:08.989826 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:07:08.989826 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:07:08.989826 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:07:08.989826 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:07:08.990694 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:07:08.990694 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:07:08.990694 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:07:08.990694 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:07:08.990694 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:07:08.990694 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:07:08.990694 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:07:08.990694 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:07:08.990694 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:07:08.990694 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:07:08.990694 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:07:08.990694 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 24 17:07:08.990694 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:07:08.990694 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:07:08.990694 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:07:08.990694 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:07:08.991579 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:07:08.991579 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:07:08.991579 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:07:08.991579 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:07:08.991579 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:07:08.991579 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:07:08.991579 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:07:08.991579 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:07:08.991579 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:07:08.991579 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:07:08.991579 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 17:07:08.991579 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:07:08.991579 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 38/78] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Aug 24 17:07:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:07:08.992200 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cc4470-03dc-4e02-ad49-039b8e1183bb None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:09.075411 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e588cda-05b1-48d3-946b-68c7200cabcd None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:09.093593 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:09.099365 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cc4470-03dc-4e02-ad49-039b8e1183bb None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:09.110050 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e88d54-e0c3-4849-acd7-f874a6eb018d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:09.110718 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e88d54-e0c3-4849-acd7-f874a6eb018d None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:09.111101 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e88d54-e0c3-4849-acd7-f874a6eb018d None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:09.111400 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e88d54-e0c3-4849-acd7-f874a6eb018d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:09.121394 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e88d54-e0c3-4849-acd7-f874a6eb018d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:07:09.126447 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-10e88d54-e0c3-4849-acd7-f874a6eb018d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:07:09.126447 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:07:09.126447 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:07:09.126447 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:07:09.126447 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:07:09.126447 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:07:09.126447 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:07:09.126447 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:07:09.126447 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:07:09.126447 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:07:09.126447 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:07:09.126447 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:07:09.126447 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:07:09.126447 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:07:09.126447 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:07:09.126447 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:07:09.127512 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:07:09.127512 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:07:09.127512 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:07:09.127512 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:07:09.127512 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:07:09.127512 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:07:09.127512 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:07:09.127512 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:07:09.127512 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:07:09.127512 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:07:09.127512 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:07:09.127512 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application with Lock( Aug 24 17:07:09.127512 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:07:09.127512 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:07:09.127512 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:07:09.127512 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:07:09.131722 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:07:09.131722 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:07:09.131722 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:07:09.131722 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:07:09.131722 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:07:09.131722 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:07:09.131722 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:07:09.131722 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:07:09.131722 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:07:09.131722 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:07:09.131722 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:07:09.131722 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:07:09.131722 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 41/79] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 24 17:07:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:07:09.131722 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e88d54-e0c3-4849-acd7-f874a6eb018d None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:09.140772 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f681a95-7681-43e9-b90e-3e53a4a38fba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:09.141446 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f681a95-7681-43e9-b90e-3e53a4a38fba None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:09.141727 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f681a95-7681-43e9-b90e-3e53a4a38fba None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:09.141998 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f681a95-7681-43e9-b90e-3e53a4a38fba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:09.154157 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:09.208865 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f681a95-7681-43e9-b90e-3e53a4a38fba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:09.210335 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f681a95-7681-43e9-b90e-3e53a4a38fba None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:09.228962 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e88d54-e0c3-4849-acd7-f874a6eb018d None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:09.262075 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17598412-2ee9-482d-b155-d56bb1d4a985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:09.262688 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17598412-2ee9-482d-b155-d56bb1d4a985 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:09.263003 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17598412-2ee9-482d-b155-d56bb1d4a985 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:09.263287 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17598412-2ee9-482d-b155-d56bb1d4a985 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:09.302678 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:09.310319 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 39/80] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 24 17:07:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 17:07:09.311713 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f681a95-7681-43e9-b90e-3e53a4a38fba None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:09.330463 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17598412-2ee9-482d-b155-d56bb1d4a985 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:09.331991 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17598412-2ee9-482d-b155-d56bb1d4a985 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:09.354571 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:09.408268 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f681a95-7681-43e9-b90e-3e53a4a38fba None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:09.428811 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 42/81] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 24 17:07:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 17:07:09.429745 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17598412-2ee9-482d-b155-d56bb1d4a985 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:09.444234 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749fe906-697d-4d76-a656-940bb5273d7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:09.444848 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749fe906-697d-4d76-a656-940bb5273d7b None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:09.445131 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749fe906-697d-4d76-a656-940bb5273d7b None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:09.445398 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749fe906-697d-4d76-a656-940bb5273d7b None admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:09.446063 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749fe906-697d-4d76-a656-940bb5273d7b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:07:09.446366 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749fe906-697d-4d76-a656-940bb5273d7b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:09.447795 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749fe906-697d-4d76-a656-940bb5273d7b None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:09.456091 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 40/82] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 24 17:07:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:09.456875 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749fe906-697d-4d76-a656-940bb5273d7b None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:09.514912 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:09.533253 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17598412-2ee9-482d-b155-d56bb1d4a985 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:09.556840 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:09.558274 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749fe906-697d-4d76-a656-940bb5273d7b None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:09.568017 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b9587b-aabe-4808-9216-8cb37aad35e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:09.568644 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b9587b-aabe-4808-9216-8cb37aad35e1 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:09.568929 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b9587b-aabe-4808-9216-8cb37aad35e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:09.569198 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b9587b-aabe-4808-9216-8cb37aad35e1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:09.570297 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b9587b-aabe-4808-9216-8cb37aad35e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:09.571852 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b9587b-aabe-4808-9216-8cb37aad35e1 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:09.573407 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 43/83] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 24 17:07:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:09.574165 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b9587b-aabe-4808-9216-8cb37aad35e1 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:09.597156 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1313bf6-b427-4cff-9cb8-e545cbb307ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:09.597700 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1313bf6-b427-4cff-9cb8-e545cbb307ed None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:09.597981 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1313bf6-b427-4cff-9cb8-e545cbb307ed None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:09.598252 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1313bf6-b427-4cff-9cb8-e545cbb307ed None admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:09.598937 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1313bf6-b427-4cff-9cb8-e545cbb307ed None admin] RBAC: Adding query filter params (name=member) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:07:09.599244 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1313bf6-b427-4cff-9cb8-e545cbb307ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:09.600684 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1313bf6-b427-4cff-9cb8-e545cbb307ed None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:09.609370 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 41/84] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Aug 24 17:07:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:09.614011 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1313bf6-b427-4cff-9cb8-e545cbb307ed None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:09.673977 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b9587b-aabe-4808-9216-8cb37aad35e1 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:09.706571 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002b35c2-32b2-4ac6-a726-eccceb454ad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:09.707255 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002b35c2-32b2-4ac6-a726-eccceb454ad9 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:09.707542 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002b35c2-32b2-4ac6-a726-eccceb454ad9 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:09.707826 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002b35c2-32b2-4ac6-a726-eccceb454ad9 None admin] PATH_INFO: `/v3/projects/9161a24523d545a8939fb56c3c930122` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:09.708951 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002b35c2-32b2-4ac6-a726-eccceb454ad9 None admin] RBAC: Authorizing `identity:get_project(project_id=9161a24523d545a8939fb56c3c930122)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:09.710405 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1313bf6-b427-4cff-9cb8-e545cbb307ed None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:09.710818 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002b35c2-32b2-4ac6-a726-eccceb454ad9 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:09.712061 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 44/85] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 24 17:07:09 2022] GET /identity/v3/projects/9161a24523d545a8939fb56c3c930122 => generated 298 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:09.712835 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002b35c2-32b2-4ac6-a726-eccceb454ad9 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:09.730967 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:09.745566 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748c3af5-320d-4edf-a2db-5b6358621ea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:09.746203 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748c3af5-320d-4edf-a2db-5b6358621ea3 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:09.746492 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748c3af5-320d-4edf-a2db-5b6358621ea3 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:09.746771 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748c3af5-320d-4edf-a2db-5b6358621ea3 None admin] PATH_INFO: `/v3/projects/9161a24523d545a8939fb56c3c930122` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:09.747935 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748c3af5-320d-4edf-a2db-5b6358621ea3 None admin] RBAC: Authorizing `identity:get_project(project_id=9161a24523d545a8939fb56c3c930122)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:09.749441 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748c3af5-320d-4edf-a2db-5b6358621ea3 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:09.751367 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 42/86] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 24 17:07:09 2022] GET /identity/v3/projects/9161a24523d545a8939fb56c3c930122 => generated 298 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:09.752098 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748c3af5-320d-4edf-a2db-5b6358621ea3 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:09.779039 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:09.820145 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002b35c2-32b2-4ac6-a726-eccceb454ad9 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:09.852263 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748c3af5-320d-4edf-a2db-5b6358621ea3 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:09.852520 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5fc678-70bd-4881-8a41-8eafcb7b8bec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:09.853151 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5fc678-70bd-4881-8a41-8eafcb7b8bec None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:09.853431 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5fc678-70bd-4881-8a41-8eafcb7b8bec None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:09.853708 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5fc678-70bd-4881-8a41-8eafcb7b8bec None admin] PATH_INFO: `/v3/users/bd42523e33c34e618cc8f1f75df2b677` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:09.854919 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5fc678-70bd-4881-8a41-8eafcb7b8bec None admin] RBAC: Authorizing `identity:get_user(user_id=bd42523e33c34e618cc8f1f75df2b677)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:09.856406 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5fc678-70bd-4881-8a41-8eafcb7b8bec None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:09.858776 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 45/87] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 24 17:07:09 2022] GET /identity/v3/users/bd42523e33c34e618cc8f1f75df2b677 => generated 275 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:09.859838 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5fc678-70bd-4881-8a41-8eafcb7b8bec None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:09.890531 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75468e4e-1291-4e6e-9c96-5cbfbff4118c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:09.891179 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75468e4e-1291-4e6e-9c96-5cbfbff4118c None admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:09.891472 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75468e4e-1291-4e6e-9c96-5cbfbff4118c None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:09.891748 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75468e4e-1291-4e6e-9c96-5cbfbff4118c None admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:09.892396 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75468e4e-1291-4e6e-9c96-5cbfbff4118c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:09.893837 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75468e4e-1291-4e6e-9c96-5cbfbff4118c None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:09.966465 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5fc678-70bd-4881-8a41-8eafcb7b8bec None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:09.972733 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 43/88] 10.222.0.18 () {60 vars in 1132 bytes} [Wed Aug 24 17:07:09 2022] POST /identity/v3/users => generated 289 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:07:09.973569 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75468e4e-1291-4e6e-9c96-5cbfbff4118c None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:09.995811 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:10.005393 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc291b20-ce5e-4b98-aec4-f59dcebb4978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:10.006194 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc291b20-ce5e-4b98-aec4-f59dcebb4978 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:10.006475 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc291b20-ce5e-4b98-aec4-f59dcebb4978 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:10.006752 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc291b20-ce5e-4b98-aec4-f59dcebb4978 None admin] PATH_INFO: `/v3/users/bd42523e33c34e618cc8f1f75df2b677` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:10.008069 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc291b20-ce5e-4b98-aec4-f59dcebb4978 None admin] RBAC: Authorizing `identity:get_user(user_id=bd42523e33c34e618cc8f1f75df2b677)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:10.009539 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc291b20-ce5e-4b98-aec4-f59dcebb4978 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:10.011226 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 46/89] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 24 17:07:09 2022] GET /identity/v3/users/bd42523e33c34e618cc8f1f75df2b677 => generated 275 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:10.011226 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc291b20-ce5e-4b98-aec4-f59dcebb4978 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:10.039583 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:10.099344 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75468e4e-1291-4e6e-9c96-5cbfbff4118c None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:10.137669 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676859d4-8c11-420c-acce-abf5c3d031fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:10.138419 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676859d4-8c11-420c-acce-abf5c3d031fd None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:10.138524 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676859d4-8c11-420c-acce-abf5c3d031fd None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:10.138749 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676859d4-8c11-420c-acce-abf5c3d031fd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:10.140175 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676859d4-8c11-420c-acce-abf5c3d031fd None admin] RBAC: Adding query filter params (role.id=60dc7e16aee746d5a3e8aa27d7d8525c, scope.project.id=9161a24523d545a8939fb56c3c930122, user.id=bd42523e33c34e618cc8f1f75df2b677) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:07:10.140488 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676859d4-8c11-420c-acce-abf5c3d031fd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:10.141883 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc291b20-ce5e-4b98-aec4-f59dcebb4978 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:10.142817 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676859d4-8c11-420c-acce-abf5c3d031fd None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:10.151744 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 44/90] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 24 17:07:09 2022] GET /identity/v3/role_assignments?user.id=bd42523e33c34e618cc8f1f75df2b677&scope.project.id=9161a24523d545a8939fb56c3c930122&role.id=60dc7e16aee746d5a3e8aa27d7d8525c => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:10.151744 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676859d4-8c11-420c-acce-abf5c3d031fd None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:10.183911 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06df911-b049-44b9-be70-e5d79dd631a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:10.184776 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06df911-b049-44b9-be70-e5d79dd631a8 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:10.185198 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06df911-b049-44b9-be70-e5d79dd631a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:10.185645 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06df911-b049-44b9-be70-e5d79dd631a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:10.186392 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06df911-b049-44b9-be70-e5d79dd631a8 None admin] RBAC: Adding query filter params (role.id=c5346bc54abf445f9900edf94264c850, scope.project.id=9161a24523d545a8939fb56c3c930122, user.id=20b73f4d8b424258a870e87b04c9a532) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:07:10.186702 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06df911-b049-44b9-be70-e5d79dd631a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:10.188228 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06df911-b049-44b9-be70-e5d79dd631a8 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:10.202435 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 47/91] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 24 17:07:10 2022] GET /identity/v3/role_assignments?user.id=20b73f4d8b424258a870e87b04c9a532&scope.project.id=9161a24523d545a8939fb56c3c930122&role.id=c5346bc54abf445f9900edf94264c850 => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:10.202435 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06df911-b049-44b9-be70-e5d79dd631a8 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:10.285066 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:10.294290 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676859d4-8c11-420c-acce-abf5c3d031fd None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:10.307624 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:10.342940 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99dd3c4-6a1d-4a49-811c-9fe7225852b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:10.343916 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99dd3c4-6a1d-4a49-811c-9fe7225852b5 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:10.344524 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99dd3c4-6a1d-4a49-811c-9fe7225852b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:10.344983 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99dd3c4-6a1d-4a49-811c-9fe7225852b5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:10.351458 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06df911-b049-44b9-be70-e5d79dd631a8 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:10.391777 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caa0a37-0bae-4e42-94b8-bcdbbf0e27c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:10.392330 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caa0a37-0bae-4e42-94b8-bcdbbf0e27c1 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:10.392616 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caa0a37-0bae-4e42-94b8-bcdbbf0e27c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:10.392895 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caa0a37-0bae-4e42-94b8-bcdbbf0e27c1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:10.481953 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99dd3c4-6a1d-4a49-811c-9fe7225852b5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:10.482594 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4caa0a37-0bae-4e42-94b8-bcdbbf0e27c1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:10.484610 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99dd3c4-6a1d-4a49-811c-9fe7225852b5 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:10.484725 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4caa0a37-0bae-4e42-94b8-bcdbbf0e27c1 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:10.570026 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:10.647108 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 48/92] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 24 17:07:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 17:07:10.652192 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caa0a37-0bae-4e42-94b8-bcdbbf0e27c1 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:10.655313 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 45/93] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 24 17:07:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 503 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 17:07:10.657488 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99dd3c4-6a1d-4a49-811c-9fe7225852b5 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:10.674662 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:10.795842 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99dd3c4-6a1d-4a49-811c-9fe7225852b5 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:10.836337 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262eb468-5804-49ee-89e0-3f8d1d902e4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:10.837248 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262eb468-5804-49ee-89e0-3f8d1d902e4e None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:10.837669 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262eb468-5804-49ee-89e0-3f8d1d902e4e None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:10.838078 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262eb468-5804-49ee-89e0-3f8d1d902e4e None admin] PATH_INFO: `/v3/projects/9161a24523d545a8939fb56c3c930122` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:10.843634 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262eb468-5804-49ee-89e0-3f8d1d902e4e None admin] RBAC: Authorizing `identity:get_project(project_id=9161a24523d545a8939fb56c3c930122)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:10.845814 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262eb468-5804-49ee-89e0-3f8d1d902e4e None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:10.847705 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 46/94] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 24 17:07:10 2022] GET /identity/v3/projects/9161a24523d545a8939fb56c3c930122 => generated 298 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:10.856557 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262eb468-5804-49ee-89e0-3f8d1d902e4e None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:10.924941 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caa0a37-0bae-4e42-94b8-bcdbbf0e27c1 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:11.012604 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d10511-9a97-46d7-b8be-78101de0c564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:11.013257 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d10511-9a97-46d7-b8be-78101de0c564 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:11.013543 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d10511-9a97-46d7-b8be-78101de0c564 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:11.013824 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d10511-9a97-46d7-b8be-78101de0c564 None admin] PATH_INFO: `/v3/projects/d00aeeaba4ee4ac696756e040b6b0585` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:11.015021 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d10511-9a97-46d7-b8be-78101de0c564 None admin] RBAC: Authorizing `identity:get_project(project_id=d00aeeaba4ee4ac696756e040b6b0585)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:11.016472 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d10511-9a97-46d7-b8be-78101de0c564 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:11.023339 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 49/95] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 24 17:07:10 2022] GET /identity/v3/projects/d00aeeaba4ee4ac696756e040b6b0585 => generated 312 bytes in 371 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:11.030204 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d10511-9a97-46d7-b8be-78101de0c564 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:11.063818 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:11.092816 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:11.121423 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262eb468-5804-49ee-89e0-3f8d1d902e4e None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:11.203698 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe7ea8c-6486-4b95-bc08-a32c7f42a76b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:11.204231 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe7ea8c-6486-4b95-bc08-a32c7f42a76b None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:11.204514 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe7ea8c-6486-4b95-bc08-a32c7f42a76b None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:11.204970 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe7ea8c-6486-4b95-bc08-a32c7f42a76b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:11.205710 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe7ea8c-6486-4b95-bc08-a32c7f42a76b None admin] RBAC: Adding query filter params (role.id=af19a7a2fae5422da5bd7daea4bbfccf, scope.project.id=9161a24523d545a8939fb56c3c930122, user.id=bd42523e33c34e618cc8f1f75df2b677) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:07:11.206022 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe7ea8c-6486-4b95-bc08-a32c7f42a76b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:11.207512 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe7ea8c-6486-4b95-bc08-a32c7f42a76b None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:11.222166 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 47/96] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 24 17:07:10 2022] GET /identity/v3/role_assignments?user.id=bd42523e33c34e618cc8f1f75df2b677&scope.project.id=9161a24523d545a8939fb56c3c930122&role.id=af19a7a2fae5422da5bd7daea4bbfccf => generated 259 bytes in 366 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:11.262174 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d10511-9a97-46d7-b8be-78101de0c564 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:11.325835 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafce7c4-6b2a-43a1-a90f-671d3c7640cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:11.326775 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafce7c4-6b2a-43a1-a90f-671d3c7640cb None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:11.327237 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafce7c4-6b2a-43a1-a90f-671d3c7640cb None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:11.327652 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafce7c4-6b2a-43a1-a90f-671d3c7640cb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:11.328819 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafce7c4-6b2a-43a1-a90f-671d3c7640cb None admin] RBAC: Adding query filter params (role.id=60dc7e16aee746d5a3e8aa27d7d8525c, scope.project.id=d00aeeaba4ee4ac696756e040b6b0585, user.id=bd42523e33c34e618cc8f1f75df2b677) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:07:11.329289 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafce7c4-6b2a-43a1-a90f-671d3c7640cb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:11.340278 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafce7c4-6b2a-43a1-a90f-671d3c7640cb None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:11.340523 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 50/97] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 24 17:07:11 2022] GET /identity/v3/role_assignments?user.id=bd42523e33c34e618cc8f1f75df2b677&scope.project.id=d00aeeaba4ee4ac696756e040b6b0585&role.id=60dc7e16aee746d5a3e8aa27d7d8525c => generated 259 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:12.776611 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc363879-625a-481c-9d83-8c6eed544436 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:12.777109 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc363879-625a-481c-9d83-8c6eed544436 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:12.777211 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc363879-625a-481c-9d83-8c6eed544436 None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:12.778650 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 48/98] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:07:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:07:12.784165 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0975cc91-05ef-4233-85aa-e40dcda9620b None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:12.784997 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0975cc91-05ef-4233-85aa-e40dcda9620b None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:12.785620 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0975cc91-05ef-4233-85aa-e40dcda9620b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:12.805636 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-0975cc91-05ef-4233-85aa-e40dcda9620b None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:07:12.807184 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-0975cc91-05ef-4233-85aa-e40dcda9620b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:07:12.900354 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 51/99] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:07:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:07:12.934736 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9792c82c-5916-49c9-815e-7a2b287690c5 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:12.935357 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9792c82c-5916-49c9-815e-7a2b287690c5 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:12.936484 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9792c82c-5916-49c9-815e-7a2b287690c5 None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:12.938028 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 49/100] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:07:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:07:12.943399 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aeefdc-dc95-475d-8650-0edf2b4ad3fc None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:12.943910 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aeefdc-dc95-475d-8650-0edf2b4ad3fc None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:12.944648 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aeefdc-dc95-475d-8650-0edf2b4ad3fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:12.969478 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-f0aeefdc-dc95-475d-8650-0edf2b4ad3fc None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:07:12.971070 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-f0aeefdc-dc95-475d-8650-0edf2b4ad3fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:07:12.973143 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cf8ad1-8736-4281-a111-6cd6594b5c3b None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:12.973624 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cf8ad1-8736-4281-a111-6cd6594b5c3b None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:12.974088 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cf8ad1-8736-4281-a111-6cd6594b5c3b None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:12.975652 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 50/101] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:07:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 17:07:12.981605 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf499431-4c1e-4231-b4da-2384e1acf206 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:12.982152 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf499431-4c1e-4231-b4da-2384e1acf206 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:12.982663 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf499431-4c1e-4231-b4da-2384e1acf206 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:13.011719 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-cf499431-4c1e-4231-b4da-2384e1acf206 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:07:13.013310 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-cf499431-4c1e-4231-b4da-2384e1acf206 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:07:13.023252 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:13.115448 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 52/102] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:07:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:07:13.136344 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35676b4d-0032-4019-8406-13eb4639ed77 None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:13.136632 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35676b4d-0032-4019-8406-13eb4639ed77 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:13.136910 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35676b4d-0032-4019-8406-13eb4639ed77 None None] PATH_INFO: `/` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:13.137435 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 51/103] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:07:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:07:13.138355 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 53/104] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:07:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 17:07:13.144998 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e961dd-087d-464d-88de-1c64652b92c2 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:13.145290 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e961dd-087d-464d-88de-1c64652b92c2 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:13.145572 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e961dd-087d-464d-88de-1c64652b92c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:13.183884 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-24e961dd-087d-464d-88de-1c64652b92c2 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:07:13.185074 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-24e961dd-087d-464d-88de-1c64652b92c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:07:13.258812 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:13.267828 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 52/105] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:07:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 17:07:13.402631 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35676b4d-0032-4019-8406-13eb4639ed77 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:13.523597 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35676b4d-0032-4019-8406-13eb4639ed77 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:13.582406 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ec87bb-7686-4ea2-af3f-6afb79cbe681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:13.583463 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ec87bb-7686-4ea2-af3f-6afb79cbe681 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:13.583947 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ec87bb-7686-4ea2-af3f-6afb79cbe681 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:13.584475 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ec87bb-7686-4ea2-af3f-6afb79cbe681 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:13.585844 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ec87bb-7686-4ea2-af3f-6afb79cbe681 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:13.587656 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ec87bb-7686-4ea2-af3f-6afb79cbe681 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:13.589760 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 54/106] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 24 17:07:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:13.600006 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e961dd-087d-464d-88de-1c64652b92c2 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:13.612540 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ec87bb-7686-4ea2-af3f-6afb79cbe681 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:13.638956 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:13.739328 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ec87bb-7686-4ea2-af3f-6afb79cbe681 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:13.752609 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e961dd-087d-464d-88de-1c64652b92c2 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:13.784330 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25605ed6-834f-46ef-9269-47437190d6e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:13.785218 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25605ed6-834f-46ef-9269-47437190d6e2 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:13.785502 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25605ed6-834f-46ef-9269-47437190d6e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:13.785807 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25605ed6-834f-46ef-9269-47437190d6e2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:13.791057 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e0da45-df59-4708-9c39-0613fb9a3e9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:13.791840 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e0da45-df59-4708-9c39-0613fb9a3e9c None admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:13.792223 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e0da45-df59-4708-9c39-0613fb9a3e9c None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:13.792754 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e0da45-df59-4708-9c39-0613fb9a3e9c None admin] PATH_INFO: `/v3/projects` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:13.793497 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e0da45-df59-4708-9c39-0613fb9a3e9c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:13.795100 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e0da45-df59-4708-9c39-0613fb9a3e9c None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:13.797882 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25605ed6-834f-46ef-9269-47437190d6e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:07:13.803371 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-25605ed6-834f-46ef-9269-47437190d6e2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:07:13.803371 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:07:13.803371 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:07:13.803371 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:07:13.803371 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:07:13.803371 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:07:13.803371 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:07:13.803371 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:07:13.803371 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:07:13.803371 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:07:13.803371 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:07:13.803371 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:07:13.803371 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:07:13.803371 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:07:13.803371 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:07:13.803371 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:07:13.804895 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:07:13.804895 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:07:13.804895 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:07:13.804895 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:07:13.804895 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:07:13.804895 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:07:13.804895 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:07:13.804895 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:07:13.804895 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:07:13.804895 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:07:13.804895 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:07:13.804895 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application with Lock( Aug 24 17:07:13.804895 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:07:13.804895 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:07:13.804895 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:07:13.804895 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:07:13.806723 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:07:13.806723 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:07:13.806723 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:07:13.806723 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:07:13.806723 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:07:13.806723 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:07:13.806723 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:07:13.806723 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:07:13.806723 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:07:13.806723 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:07:13.806723 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:07:13.806723 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:07:13.806723 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 55/107] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 24 17:07:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:07:13.806723 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25605ed6-834f-46ef-9269-47437190d6e2 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:13.891869 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:13.903497 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 53/108] 10.222.0.18 () {60 vars in 1141 bytes} [Wed Aug 24 17:07:13 2022] POST /identity/v3/projects => generated 302 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:07:13.904899 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e0da45-df59-4708-9c39-0613fb9a3e9c None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:13.946032 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25605ed6-834f-46ef-9269-47437190d6e2 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:14.030586 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a2411f-8406-4924-b7d6-88b011f57279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:14.034070 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a2411f-8406-4924-b7d6-88b011f57279 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:14.034393 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a2411f-8406-4924-b7d6-88b011f57279 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:14.034673 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a2411f-8406-4924-b7d6-88b011f57279 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:14.037581 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:14.060577 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a2411f-8406-4924-b7d6-88b011f57279 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:07:14.066702 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-41a2411f-8406-4924-b7d6-88b011f57279 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 17:07:14.066702 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:07:14.066702 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:07:14.066702 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:07:14.066702 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:07:14.066702 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:07:14.066702 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:07:14.066702 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:07:14.066702 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:07:14.066702 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:07:14.066702 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:07:14.066702 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:07:14.066702 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:07:14.066702 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:07:14.066702 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:07:14.066702 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:07:14.067588 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:07:14.067588 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:07:14.067588 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:07:14.067588 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:07:14.067588 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:07:14.067588 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:07:14.067588 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:07:14.067588 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:07:14.067588 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:07:14.067588 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:07:14.067588 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:07:14.067588 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application with Lock( Aug 24 17:07:14.067588 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:07:14.067588 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:07:14.067588 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:07:14.067588 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:07:14.068361 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:07:14.068361 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:07:14.068361 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:07:14.068361 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:07:14.068361 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:07:14.068361 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:07:14.068361 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:07:14.068361 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:07:14.068361 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:07:14.068361 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:07:14.068361 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 17:07:14.068361 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:07:14.068361 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 56/109] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 24 17:07:13 2022] GET /identity/v3/roles/reader => generated 84 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:07:14.069280 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a2411f-8406-4924-b7d6-88b011f57279 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:14.116041 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e0da45-df59-4708-9c39-0613fb9a3e9c None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:14.153787 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a72adf-2b4a-4b44-9521-03eb9e681c2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:14.155426 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a72adf-2b4a-4b44-9521-03eb9e681c2c None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:14.155910 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a72adf-2b4a-4b44-9521-03eb9e681c2c None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:14.156464 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a72adf-2b4a-4b44-9521-03eb9e681c2c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:14.170153 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a72adf-2b4a-4b44-9521-03eb9e681c2c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:07:14.176008 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-87a72adf-2b4a-4b44-9521-03eb9e681c2c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:07:14.176008 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:07:14.176008 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:07:14.176008 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:07:14.176008 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:07:14.176008 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:07:14.176008 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:07:14.176008 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:07:14.176008 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:07:14.176008 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:07:14.176008 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:07:14.176008 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:07:14.176008 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:07:14.176008 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:07:14.176008 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:07:14.176008 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:07:14.176870 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:07:14.176870 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:07:14.176870 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:07:14.176870 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:07:14.176870 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:07:14.176870 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:07:14.176870 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:07:14.176870 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:07:14.176870 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:07:14.176870 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:07:14.176870 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:07:14.176870 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 24 17:07:14.176870 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:07:14.176870 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:07:14.176870 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:07:14.176870 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:07:14.177636 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:07:14.177636 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:07:14.177636 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:07:14.177636 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:07:14.177636 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:07:14.177636 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:07:14.177636 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:07:14.177636 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:07:14.177636 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:07:14.177636 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:07:14.177636 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:07:14.177636 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:07:14.177636 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 54/110] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Aug 24 17:07:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:07:14.178366 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a72adf-2b4a-4b44-9521-03eb9e681c2c None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:14.198467 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a2411f-8406-4924-b7d6-88b011f57279 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:14.236226 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2f3330-7f7f-4722-97d3-e7da464ec277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:14.236975 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2f3330-7f7f-4722-97d3-e7da464ec277 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:14.237329 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2f3330-7f7f-4722-97d3-e7da464ec277 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:14.237674 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2f3330-7f7f-4722-97d3-e7da464ec277 None admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:14.238419 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2f3330-7f7f-4722-97d3-e7da464ec277 None admin] RBAC: Adding query filter params (name=member) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:07:14.238793 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2f3330-7f7f-4722-97d3-e7da464ec277 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:14.240426 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2f3330-7f7f-4722-97d3-e7da464ec277 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:14.246718 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:14.251342 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 57/111] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Aug 24 17:07:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:14.252629 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd59a927-c1e2-45d9-8b18-2afeb56e3284 None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:14.257530 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd59a927-c1e2-45d9-8b18-2afeb56e3284 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:14.257864 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd59a927-c1e2-45d9-8b18-2afeb56e3284 None None] PATH_INFO: `/` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:14.259128 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 58/112] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:07:14 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:07:14.263103 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd59a927-c1e2-45d9-8b18-2afeb56e3284 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:14.313738 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a72adf-2b4a-4b44-9521-03eb9e681c2c None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:14.327781 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:14.350760 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6aa17c-c14d-461b-9b6e-f8f8b80dae4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:14.351406 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6aa17c-c14d-461b-9b6e-f8f8b80dae4f None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:14.351689 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6aa17c-c14d-461b-9b6e-f8f8b80dae4f None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:14.351965 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6aa17c-c14d-461b-9b6e-f8f8b80dae4f None admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:14.352640 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6aa17c-c14d-461b-9b6e-f8f8b80dae4f None admin] RBAC: Adding query filter params (name=reader) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:07:14.352946 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6aa17c-c14d-461b-9b6e-f8f8b80dae4f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:14.354344 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6aa17c-c14d-461b-9b6e-f8f8b80dae4f None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:14.363924 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 55/113] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Aug 24 17:07:14 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:14.364935 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde7fe27-8ca8-41b0-9845-6ffe1750780b None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:14.365226 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde7fe27-8ca8-41b0-9845-6ffe1750780b None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:14.365505 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde7fe27-8ca8-41b0-9845-6ffe1750780b None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:14.366700 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 56/114] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:07:14 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:07:14.367583 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde7fe27-8ca8-41b0-9845-6ffe1750780b None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:14.377115 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd59a927-c1e2-45d9-8b18-2afeb56e3284 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:14.415644 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9e56a8-7524-419a-9abc-253ed872c753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:14.416270 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9e56a8-7524-419a-9abc-253ed872c753 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:14.416558 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9e56a8-7524-419a-9abc-253ed872c753 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:14.416837 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9e56a8-7524-419a-9abc-253ed872c753 None admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:14.417510 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9e56a8-7524-419a-9abc-253ed872c753 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:07:14.418218 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9e56a8-7524-419a-9abc-253ed872c753 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:14.419670 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9e56a8-7524-419a-9abc-253ed872c753 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:14.429313 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 59/115] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 24 17:07:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:14.430507 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe3d4ae-3a35-4390-a607-51194e1e3416 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:14.430795 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe3d4ae-3a35-4390-a607-51194e1e3416 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:14.431101 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe3d4ae-3a35-4390-a607-51194e1e3416 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:14.449186 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-cbe3d4ae-3a35-4390-a607-51194e1e3416 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:07:14.450407 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-cbe3d4ae-3a35-4390-a607-51194e1e3416 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:07:14.487889 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde7fe27-8ca8-41b0-9845-6ffe1750780b None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:14.512443 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:14.531717 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 60/116] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:07:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:07:14.531883 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ebfb5d-ea3a-4d1c-8f82-1cf0977475ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:14.531976 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe3d4ae-3a35-4390-a607-51194e1e3416 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:14.532046 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ebfb5d-ea3a-4d1c-8f82-1cf0977475ab None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:14.532046 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ebfb5d-ea3a-4d1c-8f82-1cf0977475ab None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:14.532046 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ebfb5d-ea3a-4d1c-8f82-1cf0977475ab None admin] PATH_INFO: `/v3/users/bd42523e33c34e618cc8f1f75df2b677` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:14.532046 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ebfb5d-ea3a-4d1c-8f82-1cf0977475ab None admin] RBAC: Authorizing `identity:get_user(user_id=bd42523e33c34e618cc8f1f75df2b677)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:14.532046 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ebfb5d-ea3a-4d1c-8f82-1cf0977475ab None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:14.532046 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 57/117] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 24 17:07:14 2022] GET /identity/v3/users/bd42523e33c34e618cc8f1f75df2b677 => generated 275 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:14.537451 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81b0f6a-a1cb-4c64-a6b7-835e0495eaf8 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:14.537745 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81b0f6a-a1cb-4c64-a6b7-835e0495eaf8 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:14.538029 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81b0f6a-a1cb-4c64-a6b7-835e0495eaf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:14.557850 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-b81b0f6a-a1cb-4c64-a6b7-835e0495eaf8 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:07:14.559252 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-b81b0f6a-a1cb-4c64-a6b7-835e0495eaf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:07:14.607279 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:14.643215 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 58/118] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:07:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:07:14.643955 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81b0f6a-a1cb-4c64-a6b7-835e0495eaf8 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:14.647961 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe3d4ae-3a35-4390-a607-51194e1e3416 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:14.721437 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7648537e-232b-4d7e-848d-8c91d9d299cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:14.722083 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7648537e-232b-4d7e-848d-8c91d9d299cf None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:14.722363 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7648537e-232b-4d7e-848d-8c91d9d299cf None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:14.722632 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7648537e-232b-4d7e-848d-8c91d9d299cf None admin] PATH_INFO: `/v3/users/8fbef5751d094b0abe9e4ca3657fea9e` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:14.751642 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7648537e-232b-4d7e-848d-8c91d9d299cf None admin] RBAC: Authorizing `identity:get_user(user_id=8fbef5751d094b0abe9e4ca3657fea9e)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:14.754549 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7648537e-232b-4d7e-848d-8c91d9d299cf None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:14.754868 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 61/119] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 24 17:07:14 2022] GET /identity/v3/users/8fbef5751d094b0abe9e4ca3657fea9e => generated 289 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:14.755704 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7648537e-232b-4d7e-848d-8c91d9d299cf None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:14.797543 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81b0f6a-a1cb-4c64-a6b7-835e0495eaf8 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:14.839356 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8447f5e3-9514-46a7-aab6-dd1969eb8141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:14.840000 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8447f5e3-9514-46a7-aab6-dd1969eb8141 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:14.840280 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8447f5e3-9514-46a7-aab6-dd1969eb8141 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:14.840555 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8447f5e3-9514-46a7-aab6-dd1969eb8141 None admin] PATH_INFO: `/v3/users/20b73f4d8b424258a870e87b04c9a532` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:14.843654 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8447f5e3-9514-46a7-aab6-dd1969eb8141 None admin] RBAC: Authorizing `identity:get_user(user_id=20b73f4d8b424258a870e87b04c9a532)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:14.845134 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8447f5e3-9514-46a7-aab6-dd1969eb8141 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:14.846730 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 59/120] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 24 17:07:14 2022] GET /identity/v3/users/20b73f4d8b424258a870e87b04c9a532 => generated 247 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:14.847537 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8447f5e3-9514-46a7-aab6-dd1969eb8141 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:14.868510 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:14.903739 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:14.934802 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7648537e-232b-4d7e-848d-8c91d9d299cf None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:14.974619 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8447f5e3-9514-46a7-aab6-dd1969eb8141 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:14.976135 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411bcd2f-858a-43a5-a6cd-564030724e10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:14.976718 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411bcd2f-858a-43a5-a6cd-564030724e10 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:14.976998 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411bcd2f-858a-43a5-a6cd-564030724e10 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:14.977949 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411bcd2f-858a-43a5-a6cd-564030724e10 None admin] PATH_INFO: `/v3/projects/9161a24523d545a8939fb56c3c930122` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:14.978550 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411bcd2f-858a-43a5-a6cd-564030724e10 None admin] RBAC: Authorizing `identity:get_project(project_id=9161a24523d545a8939fb56c3c930122)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:14.980012 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411bcd2f-858a-43a5-a6cd-564030724e10 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:14.981630 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 62/121] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 24 17:07:14 2022] GET /identity/v3/projects/9161a24523d545a8939fb56c3c930122 => generated 298 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:14.982389 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411bcd2f-858a-43a5-a6cd-564030724e10 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:15.016138 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24839c9d-dd5c-41ce-b9be-3b33c7553e48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:15.016752 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24839c9d-dd5c-41ce-b9be-3b33c7553e48 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:15.017029 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24839c9d-dd5c-41ce-b9be-3b33c7553e48 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:15.017302 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24839c9d-dd5c-41ce-b9be-3b33c7553e48 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:15.089901 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24839c9d-dd5c-41ce-b9be-3b33c7553e48 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:15.091437 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24839c9d-dd5c-41ce-b9be-3b33c7553e48 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:15.097289 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411bcd2f-858a-43a5-a6cd-564030724e10 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:15.114800 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:15.132821 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bdf7be-3bb5-41ee-b7d4-f0ecd2aba1a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:15.133393 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bdf7be-3bb5-41ee-b7d4-f0ecd2aba1a2 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:15.133766 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bdf7be-3bb5-41ee-b7d4-f0ecd2aba1a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:15.134040 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bdf7be-3bb5-41ee-b7d4-f0ecd2aba1a2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:15.139392 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:15.146670 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bdf7be-3bb5-41ee-b7d4-f0ecd2aba1a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:07:15.151428 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-b4bdf7be-3bb5-41ee-b7d4-f0ecd2aba1a2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:07:15.151428 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:07:15.151428 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:07:15.151428 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:07:15.151428 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:07:15.151428 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:07:15.151428 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:07:15.151428 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:07:15.151428 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:07:15.151428 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:07:15.151428 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:07:15.151428 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:07:15.151428 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:07:15.151428 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:07:15.151428 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:07:15.151428 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:07:15.152262 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:07:15.152262 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:07:15.152262 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:07:15.152262 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:07:15.152262 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:07:15.152262 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:07:15.152262 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:07:15.152262 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:07:15.152262 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:07:15.152262 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:07:15.152262 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:07:15.152262 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application with Lock( Aug 24 17:07:15.152262 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:07:15.152262 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:07:15.152262 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:07:15.152262 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:07:15.153050 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:07:15.153050 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:07:15.153050 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:07:15.153050 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:07:15.153050 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:07:15.153050 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:07:15.153050 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:07:15.153050 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:07:15.153050 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:07:15.153050 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:07:15.153050 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:07:15.153050 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:07:15.153050 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 63/122] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 24 17:07:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:07:15.153679 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bdf7be-3bb5-41ee-b7d4-f0ecd2aba1a2 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:15.205788 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 60/123] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 24 17:07:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 17:07:15.206633 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24839c9d-dd5c-41ce-b9be-3b33c7553e48 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:15.264354 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bdf7be-3bb5-41ee-b7d4-f0ecd2aba1a2 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:15.303647 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca6af05-2462-484b-a5ce-9ccbcbe9e881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:15.304656 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca6af05-2462-484b-a5ce-9ccbcbe9e881 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:15.305247 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca6af05-2462-484b-a5ce-9ccbcbe9e881 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:15.305588 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca6af05-2462-484b-a5ce-9ccbcbe9e881 None admin] PATH_INFO: `/v3/projects/9161a24523d545a8939fb56c3c930122` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:15.306856 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca6af05-2462-484b-a5ce-9ccbcbe9e881 None admin] RBAC: Authorizing `identity:get_project(project_id=9161a24523d545a8939fb56c3c930122)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:15.308853 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca6af05-2462-484b-a5ce-9ccbcbe9e881 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:15.309691 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24839c9d-dd5c-41ce-b9be-3b33c7553e48 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:15.315850 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 64/124] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 24 17:07:15 2022] GET /identity/v3/projects/9161a24523d545a8939fb56c3c930122 => generated 298 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:15.315850 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca6af05-2462-484b-a5ce-9ccbcbe9e881 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:15.345696 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54ae7c1-675c-451a-b73a-cb4b2e72ab2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:15.346331 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54ae7c1-675c-451a-b73a-cb4b2e72ab2a None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:15.346611 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54ae7c1-675c-451a-b73a-cb4b2e72ab2a None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:15.346882 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54ae7c1-675c-451a-b73a-cb4b2e72ab2a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:15.355170 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:15.358646 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54ae7c1-675c-451a-b73a-cb4b2e72ab2a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:07:15.363410 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-a54ae7c1-675c-451a-b73a-cb4b2e72ab2a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 17:07:15.363410 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:07:15.363410 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:07:15.363410 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:07:15.363410 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:07:15.363410 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:07:15.363410 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:07:15.363410 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:07:15.363410 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:07:15.363410 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:07:15.363410 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:07:15.363410 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:07:15.363410 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:07:15.363410 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:07:15.363410 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:07:15.363410 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:07:15.364264 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:07:15.364264 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:07:15.364264 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:07:15.364264 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:07:15.364264 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:07:15.364264 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:07:15.364264 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:07:15.364264 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:07:15.364264 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:07:15.364264 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:07:15.364264 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:07:15.364264 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 24 17:07:15.364264 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:07:15.364264 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:07:15.364264 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:07:15.364264 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:07:15.365053 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:07:15.365053 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:07:15.365053 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:07:15.365053 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:07:15.365053 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:07:15.365053 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:07:15.365053 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:07:15.365053 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:07:15.365053 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:07:15.365053 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:07:15.365053 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 17:07:15.365053 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:07:15.365053 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 61/125] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Aug 24 17:07:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:07:15.366204 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54ae7c1-675c-451a-b73a-cb4b2e72ab2a None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:15.372882 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:15.429600 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca6af05-2462-484b-a5ce-9ccbcbe9e881 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:15.465481 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecb76df-9ec7-4f16-814c-dcb5f9e59016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:15.466150 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecb76df-9ec7-4f16-814c-dcb5f9e59016 None admin] REQUEST_METHOD: `PUT` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:15.466438 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecb76df-9ec7-4f16-814c-dcb5f9e59016 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:15.466783 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecb76df-9ec7-4f16-814c-dcb5f9e59016 None admin] PATH_INFO: `/v3/projects/9161a24523d545a8939fb56c3c930122/users/bd42523e33c34e618cc8f1f75df2b677/roles/60dc7e16aee746d5a3e8aa27d7d8525c` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:15.468786 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecb76df-9ec7-4f16-814c-dcb5f9e59016 None admin] RBAC: Authorizing `identity:create_grant(project_id=9161a24523d545a8939fb56c3c930122, user_id=bd42523e33c34e618cc8f1f75df2b677, role_id=60dc7e16aee746d5a3e8aa27d7d8525c)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:15.470244 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecb76df-9ec7-4f16-814c-dcb5f9e59016 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:15.478105 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54ae7c1-675c-451a-b73a-cb4b2e72ab2a None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:15.513062 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd883f9-b78b-45af-a974-6f051a8d928b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:15.513677 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd883f9-b78b-45af-a974-6f051a8d928b None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:15.513973 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd883f9-b78b-45af-a974-6f051a8d928b None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:15.514252 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd883f9-b78b-45af-a974-6f051a8d928b None admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:15.514951 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd883f9-b78b-45af-a974-6f051a8d928b None admin] RBAC: Adding query filter params (name=member) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:07:15.515268 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd883f9-b78b-45af-a974-6f051a8d928b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:15.516668 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd883f9-b78b-45af-a974-6f051a8d928b None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:15.524647 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 62/126] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Aug 24 17:07:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:15.529337 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd883f9-b78b-45af-a974-6f051a8d928b None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:15.562935 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 65/127] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Aug 24 17:07:15 2022] PUT /identity/v3/projects/9161a24523d545a8939fb56c3c930122/users/bd42523e33c34e618cc8f1f75df2b677/roles/60dc7e16aee746d5a3e8aa27d7d8525c => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:07:15.562935 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecb76df-9ec7-4f16-814c-dcb5f9e59016 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:15.602754 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:15.651883 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd883f9-b78b-45af-a974-6f051a8d928b None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:15.692692 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b15d09-655b-4983-bb6e-2eb19a26d4f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:15.693818 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b15d09-655b-4983-bb6e-2eb19a26d4f1 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:15.693818 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b15d09-655b-4983-bb6e-2eb19a26d4f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:15.694213 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b15d09-655b-4983-bb6e-2eb19a26d4f1 None admin] PATH_INFO: `/v3/projects/9161a24523d545a8939fb56c3c930122` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:15.695425 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b15d09-655b-4983-bb6e-2eb19a26d4f1 None admin] RBAC: Authorizing `identity:get_project(project_id=9161a24523d545a8939fb56c3c930122)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:15.696893 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b15d09-655b-4983-bb6e-2eb19a26d4f1 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:15.699549 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 63/128] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 24 17:07:15 2022] GET /identity/v3/projects/9161a24523d545a8939fb56c3c930122 => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:15.700308 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b15d09-655b-4983-bb6e-2eb19a26d4f1 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:15.707463 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecb76df-9ec7-4f16-814c-dcb5f9e59016 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:15.716271 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:15.748447 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72962f6f-af94-4c43-a086-d7a4589acd6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:15.749576 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72962f6f-af94-4c43-a086-d7a4589acd6b None admin] REQUEST_METHOD: `PUT` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:15.750501 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72962f6f-af94-4c43-a086-d7a4589acd6b None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:15.751507 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72962f6f-af94-4c43-a086-d7a4589acd6b None admin] PATH_INFO: `/v3/projects/9161a24523d545a8939fb56c3c930122/users/20b73f4d8b424258a870e87b04c9a532/roles/c5346bc54abf445f9900edf94264c850` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:15.755589 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72962f6f-af94-4c43-a086-d7a4589acd6b None admin] RBAC: Authorizing `identity:create_grant(project_id=9161a24523d545a8939fb56c3c930122, user_id=20b73f4d8b424258a870e87b04c9a532, role_id=c5346bc54abf445f9900edf94264c850)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:15.757638 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72962f6f-af94-4c43-a086-d7a4589acd6b None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:15.815905 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b15d09-655b-4983-bb6e-2eb19a26d4f1 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:15.846703 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:15.853591 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49b7483-1ca3-49a4-b877-2938f19c18e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:15.854216 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49b7483-1ca3-49a4-b877-2938f19c18e9 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:15.854503 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49b7483-1ca3-49a4-b877-2938f19c18e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:15.854781 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49b7483-1ca3-49a4-b877-2938f19c18e9 None admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:15.855640 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49b7483-1ca3-49a4-b877-2938f19c18e9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:07:15.855955 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49b7483-1ca3-49a4-b877-2938f19c18e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:15.857384 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49b7483-1ca3-49a4-b877-2938f19c18e9 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:15.872174 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 64/129] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 24 17:07:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:15.872174 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49b7483-1ca3-49a4-b877-2938f19c18e9 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:15.879052 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 66/130] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Aug 24 17:07:15 2022] PUT /identity/v3/projects/9161a24523d545a8939fb56c3c930122/users/20b73f4d8b424258a870e87b04c9a532/roles/c5346bc54abf445f9900edf94264c850 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:07:15.887672 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72962f6f-af94-4c43-a086-d7a4589acd6b None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:16.016495 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72962f6f-af94-4c43-a086-d7a4589acd6b None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:16.047998 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49b7483-1ca3-49a4-b877-2938f19c18e9 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:16.056301 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5f30d5-7cd4-4615-9b8f-7a9f2b07b451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:16.057293 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5f30d5-7cd4-4615-9b8f-7a9f2b07b451 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:16.057777 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5f30d5-7cd4-4615-9b8f-7a9f2b07b451 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:16.058274 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5f30d5-7cd4-4615-9b8f-7a9f2b07b451 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:16.059367 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5f30d5-7cd4-4615-9b8f-7a9f2b07b451 None admin] RBAC: Adding query filter params (role.id=7f65138ab599491194e7f91aa424ceee, scope.project.id=9161a24523d545a8939fb56c3c930122, user.id=8fbef5751d094b0abe9e4ca3657fea9e) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:07:16.059889 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5f30d5-7cd4-4615-9b8f-7a9f2b07b451 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:16.061711 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5f30d5-7cd4-4615-9b8f-7a9f2b07b451 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:16.068805 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 67/131] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 24 17:07:15 2022] GET /identity/v3/role_assignments?user.id=8fbef5751d094b0abe9e4ca3657fea9e&scope.project.id=9161a24523d545a8939fb56c3c930122&role.id=7f65138ab599491194e7f91aa424ceee => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:16.070034 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5f30d5-7cd4-4615-9b8f-7a9f2b07b451 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:16.089314 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284bcbf1-d799-44e5-bf97-fbfb3ab51d41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:16.090073 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284bcbf1-d799-44e5-bf97-fbfb3ab51d41 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:16.090362 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284bcbf1-d799-44e5-bf97-fbfb3ab51d41 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:16.090639 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284bcbf1-d799-44e5-bf97-fbfb3ab51d41 None admin] PATH_INFO: `/v3/users/bd42523e33c34e618cc8f1f75df2b677` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:16.091939 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284bcbf1-d799-44e5-bf97-fbfb3ab51d41 None admin] RBAC: Authorizing `identity:get_user(user_id=bd42523e33c34e618cc8f1f75df2b677)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:16.094336 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284bcbf1-d799-44e5-bf97-fbfb3ab51d41 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:16.096256 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 65/132] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 24 17:07:15 2022] GET /identity/v3/users/bd42523e33c34e618cc8f1f75df2b677 => generated 275 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:16.097562 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4bd6a1-73aa-46a6-b06f-61b3360ce4ac None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:16.097851 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4bd6a1-73aa-46a6-b06f-61b3360ce4ac None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:16.098230 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4bd6a1-73aa-46a6-b06f-61b3360ce4ac None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:16.099602 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 66/133] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:07:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:07:16.103001 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4bd6a1-73aa-46a6-b06f-61b3360ce4ac None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:16.116843 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:16.203000 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:16.211089 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5f30d5-7cd4-4615-9b8f-7a9f2b07b451 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:16.262238 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4bd6a1-73aa-46a6-b06f-61b3360ce4ac None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:16.266582 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575b709d-df77-4e68-bb53-a8e9df8899c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:16.267199 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575b709d-df77-4e68-bb53-a8e9df8899c7 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:16.267489 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575b709d-df77-4e68-bb53-a8e9df8899c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:16.267772 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575b709d-df77-4e68-bb53-a8e9df8899c7 None admin] PATH_INFO: `/v3/users/bd42523e33c34e618cc8f1f75df2b677` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:16.268928 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575b709d-df77-4e68-bb53-a8e9df8899c7 None admin] RBAC: Authorizing `identity:get_user(user_id=bd42523e33c34e618cc8f1f75df2b677)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:16.270449 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575b709d-df77-4e68-bb53-a8e9df8899c7 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:16.272387 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 68/134] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 24 17:07:16 2022] GET /identity/v3/users/bd42523e33c34e618cc8f1f75df2b677 => generated 275 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:16.273589 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0beb46-8cd1-4c32-8c46-259ba602885f None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:16.273878 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0beb46-8cd1-4c32-8c46-259ba602885f None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:16.274155 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0beb46-8cd1-4c32-8c46-259ba602885f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:16.299400 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-6b0beb46-8cd1-4c32-8c46-259ba602885f None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:07:16.300599 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-6b0beb46-8cd1-4c32-8c46-259ba602885f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:07:16.314176 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01607d98-59dc-4dba-956d-4805c2c50269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:16.314832 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01607d98-59dc-4dba-956d-4805c2c50269 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:16.315157 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01607d98-59dc-4dba-956d-4805c2c50269 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:16.315436 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01607d98-59dc-4dba-956d-4805c2c50269 None admin] PATH_INFO: `/v3/projects/d00aeeaba4ee4ac696756e040b6b0585` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:16.318343 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01607d98-59dc-4dba-956d-4805c2c50269 None admin] RBAC: Authorizing `identity:get_project(project_id=d00aeeaba4ee4ac696756e040b6b0585)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:16.319967 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01607d98-59dc-4dba-956d-4805c2c50269 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:16.321867 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 67/135] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 24 17:07:16 2022] GET /identity/v3/projects/d00aeeaba4ee4ac696756e040b6b0585 => generated 312 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:16.322471 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01607d98-59dc-4dba-956d-4805c2c50269 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:16.439652 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 69/136] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:07:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:07:16.441138 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0beb46-8cd1-4c32-8c46-259ba602885f None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:16.498919 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:16.531527 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01607d98-59dc-4dba-956d-4805c2c50269 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:16.583662 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0beb46-8cd1-4c32-8c46-259ba602885f None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:16.600287 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:16.626718 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056fdecf-4892-4651-9542-f45674363c08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:16.627433 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056fdecf-4892-4651-9542-f45674363c08 None admin] REQUEST_METHOD: `PUT` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:16.627720 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056fdecf-4892-4651-9542-f45674363c08 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:16.628002 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056fdecf-4892-4651-9542-f45674363c08 None admin] PATH_INFO: `/v3/projects/d00aeeaba4ee4ac696756e040b6b0585/users/bd42523e33c34e618cc8f1f75df2b677/roles/60dc7e16aee746d5a3e8aa27d7d8525c` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:16.629926 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056fdecf-4892-4651-9542-f45674363c08 None admin] RBAC: Authorizing `identity:create_grant(project_id=d00aeeaba4ee4ac696756e040b6b0585, user_id=bd42523e33c34e618cc8f1f75df2b677, role_id=60dc7e16aee746d5a3e8aa27d7d8525c)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:16.631565 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056fdecf-4892-4651-9542-f45674363c08 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:16.635623 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd25c2a-ef30-4b06-bcec-f760f266d0e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:16.636280 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd25c2a-ef30-4b06-bcec-f760f266d0e5 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:16.636574 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd25c2a-ef30-4b06-bcec-f760f266d0e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:16.636858 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd25c2a-ef30-4b06-bcec-f760f266d0e5 None admin] PATH_INFO: `/v3/projects/9161a24523d545a8939fb56c3c930122` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:16.638056 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd25c2a-ef30-4b06-bcec-f760f266d0e5 None admin] RBAC: Authorizing `identity:get_project(project_id=9161a24523d545a8939fb56c3c930122)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:16.640078 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd25c2a-ef30-4b06-bcec-f760f266d0e5 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:16.641744 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 68/137] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 24 17:07:16 2022] GET /identity/v3/projects/9161a24523d545a8939fb56c3c930122 => generated 298 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:16.647338 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd25c2a-ef30-4b06-bcec-f760f266d0e5 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:16.700466 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 70/138] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Aug 24 17:07:16 2022] PUT /identity/v3/projects/d00aeeaba4ee4ac696756e040b6b0585/users/bd42523e33c34e618cc8f1f75df2b677/roles/60dc7e16aee746d5a3e8aa27d7d8525c => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:07:16.791336 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd25c2a-ef30-4b06-bcec-f760f266d0e5 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:16.823321 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056fdecf-4892-4651-9542-f45674363c08 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:16.829740 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60e7870-5001-47b5-892f-f2518949cc92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:16.831186 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60e7870-5001-47b5-892f-f2518949cc92 None admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:16.831661 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60e7870-5001-47b5-892f-f2518949cc92 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:16.832122 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60e7870-5001-47b5-892f-f2518949cc92 None admin] PATH_INFO: `/v3/projects/9161a24523d545a8939fb56c3c930122/users/bd42523e33c34e618cc8f1f75df2b677/roles/af19a7a2fae5422da5bd7daea4bbfccf` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:16.834567 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60e7870-5001-47b5-892f-f2518949cc92 None admin] RBAC: Authorizing `identity:create_grant(project_id=9161a24523d545a8939fb56c3c930122, user_id=bd42523e33c34e618cc8f1f75df2b677, role_id=af19a7a2fae5422da5bd7daea4bbfccf)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:16.836278 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60e7870-5001-47b5-892f-f2518949cc92 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:16.911830 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 69/139] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Aug 24 17:07:16 2022] PUT /identity/v3/projects/9161a24523d545a8939fb56c3c930122/users/bd42523e33c34e618cc8f1f75df2b677/roles/af19a7a2fae5422da5bd7daea4bbfccf => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:07:16.956443 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:16.974049 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056fdecf-4892-4651-9542-f45674363c08 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:17.018625 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc5bbc0-77fb-4348-bdd1-4f4cd01ce110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:17.019319 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc5bbc0-77fb-4348-bdd1-4f4cd01ce110 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:17.019604 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc5bbc0-77fb-4348-bdd1-4f4cd01ce110 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:17.019883 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc5bbc0-77fb-4348-bdd1-4f4cd01ce110 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:17.021041 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc5bbc0-77fb-4348-bdd1-4f4cd01ce110 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:17.022541 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc5bbc0-77fb-4348-bdd1-4f4cd01ce110 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:17.024231 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 71/140] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 24 17:07:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:17.036038 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60e7870-5001-47b5-892f-f2518949cc92 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:17.139838 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:17.175369 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60e7870-5001-47b5-892f-f2518949cc92 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:17.213923 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b938c3-e36c-47c6-9e96-8957c96c5db1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:17.214540 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b938c3-e36c-47c6-9e96-8957c96c5db1 None admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:17.214819 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b938c3-e36c-47c6-9e96-8957c96c5db1 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:17.215120 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b938c3-e36c-47c6-9e96-8957c96c5db1 None admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:17.215841 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b938c3-e36c-47c6-9e96-8957c96c5db1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:17.217409 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b938c3-e36c-47c6-9e96-8957c96c5db1 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:17.347020 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 70/141] 10.222.0.18 () {60 vars in 1132 bytes} [Wed Aug 24 17:07:17 2022] POST /identity/v3/users => generated 283 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:07:18.416803 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a322fc93-49fe-45a9-ab61-26398597b0b7 None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:18.417334 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a322fc93-49fe-45a9-ab61-26398597b0b7 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:18.417334 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a322fc93-49fe-45a9-ab61-26398597b0b7 None None] PATH_INFO: `/` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:18.418712 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 72/142] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:07:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:07:18.424384 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b970b84a-f238-4894-80d0-a9dae1417703 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:18.425214 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b970b84a-f238-4894-80d0-a9dae1417703 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:18.425696 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b970b84a-f238-4894-80d0-a9dae1417703 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:18.451126 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-b970b84a-f238-4894-80d0-a9dae1417703 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:07:18.453292 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-b970b84a-f238-4894-80d0-a9dae1417703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:07:18.528133 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 71/143] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:07:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:07:18.575137 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eec8a47-d90e-4574-bf12-32e24966063b None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:18.575429 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eec8a47-d90e-4574-bf12-32e24966063b None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:18.575707 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eec8a47-d90e-4574-bf12-32e24966063b None None] PATH_INFO: `/` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:18.577908 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 73/144] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:07:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:07:18.583184 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5c75e8-8955-4e1b-a470-9bca3a91aa58 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:18.583480 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5c75e8-8955-4e1b-a470-9bca3a91aa58 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:18.583761 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5c75e8-8955-4e1b-a470-9bca3a91aa58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:18.610395 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-de5c75e8-8955-4e1b-a470-9bca3a91aa58 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:07:18.611650 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-de5c75e8-8955-4e1b-a470-9bca3a91aa58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:07:18.658979 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:18.695546 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 72/145] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:07:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 17:07:18.737799 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4374af5c-a101-4e21-9bfa-8101907455e2 None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:18.738115 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4374af5c-a101-4e21-9bfa-8101907455e2 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:18.738421 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4374af5c-a101-4e21-9bfa-8101907455e2 None None] PATH_INFO: `/` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:18.739866 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 74/146] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:07:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:07:18.745607 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402195bb-d955-4de3-9dec-2074ebb40b93 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:18.745888 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402195bb-d955-4de3-9dec-2074ebb40b93 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:18.746172 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402195bb-d955-4de3-9dec-2074ebb40b93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:18.778679 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-402195bb-d955-4de3-9dec-2074ebb40b93 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:07:18.779955 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-402195bb-d955-4de3-9dec-2074ebb40b93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:07:18.868377 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 73/147] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:07:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:07:19.140512 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4374af5c-a101-4e21-9bfa-8101907455e2 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:19.166779 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402195bb-d955-4de3-9dec-2074ebb40b93 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:19.283927 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4374af5c-a101-4e21-9bfa-8101907455e2 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:19.342688 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9b2b59-50c9-409f-b717-4d6896897bac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:19.344089 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9b2b59-50c9-409f-b717-4d6896897bac None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:19.344526 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9b2b59-50c9-409f-b717-4d6896897bac None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:19.344949 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9b2b59-50c9-409f-b717-4d6896897bac None admin] PATH_INFO: `/v3/roles/member` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:19.363103 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402195bb-d955-4de3-9dec-2074ebb40b93 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:19.364277 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9b2b59-50c9-409f-b717-4d6896897bac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:07:19.364579 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:19.370572 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-ef9b2b59-50c9-409f-b717-4d6896897bac None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:07:19.370572 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:07:19.370572 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:07:19.370572 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:07:19.370572 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:07:19.370572 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:07:19.370572 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:07:19.370572 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:07:19.370572 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:07:19.370572 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:07:19.370572 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:07:19.370572 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:07:19.370572 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:07:19.370572 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:07:19.370572 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:07:19.370572 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:07:19.371506 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:07:19.371506 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:07:19.371506 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:07:19.371506 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:07:19.371506 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:07:19.371506 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:07:19.371506 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:07:19.371506 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:07:19.371506 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:07:19.371506 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:07:19.371506 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:07:19.371506 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application with Lock( Aug 24 17:07:19.371506 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:07:19.371506 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:07:19.371506 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:07:19.371506 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:07:19.372267 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:07:19.372267 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:07:19.372267 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:07:19.372267 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:07:19.372267 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:07:19.372267 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:07:19.372267 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:07:19.372267 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:07:19.372267 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:07:19.372267 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:07:19.372267 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:07:19.372267 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:07:19.372267 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 75/148] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 24 17:07:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:07:19.375585 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9b2b59-50c9-409f-b717-4d6896897bac None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:19.414170 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:19.434175 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a90e3aa-7be7-4ca6-a570-7d66ac3dbf51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:19.435519 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a90e3aa-7be7-4ca6-a570-7d66ac3dbf51 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:19.436034 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a90e3aa-7be7-4ca6-a570-7d66ac3dbf51 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:19.436504 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a90e3aa-7be7-4ca6-a570-7d66ac3dbf51 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:19.461155 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a90e3aa-7be7-4ca6-a570-7d66ac3dbf51 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:07:19.466369 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-9a90e3aa-7be7-4ca6-a570-7d66ac3dbf51 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:07:19.466369 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:07:19.466369 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:07:19.466369 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:07:19.466369 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:07:19.466369 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:07:19.466369 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:07:19.466369 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:07:19.466369 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:07:19.466369 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:07:19.466369 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:07:19.466369 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:07:19.466369 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:07:19.466369 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:07:19.466369 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:07:19.466369 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:07:19.467892 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:07:19.467892 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:07:19.467892 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:07:19.467892 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:07:19.467892 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:07:19.467892 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:07:19.467892 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:07:19.467892 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:07:19.467892 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:07:19.467892 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:07:19.467892 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:07:19.467892 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 24 17:07:19.467892 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:07:19.467892 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:07:19.467892 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:07:19.467892 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:07:19.468644 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:07:19.468644 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:07:19.468644 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:07:19.468644 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:07:19.468644 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:07:19.468644 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:07:19.468644 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:07:19.468644 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:07:19.468644 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:07:19.468644 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:07:19.468644 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:07:19.468644 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:07:19.471168 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 74/149] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Aug 24 17:07:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:07:19.471877 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a90e3aa-7be7-4ca6-a570-7d66ac3dbf51 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:19.565798 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9b2b59-50c9-409f-b717-4d6896897bac None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:19.622342 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b985aab-8567-4f4a-8816-d26d8308961c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:19.623018 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b985aab-8567-4f4a-8816-d26d8308961c None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:19.623831 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b985aab-8567-4f4a-8816-d26d8308961c None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:19.624105 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b985aab-8567-4f4a-8816-d26d8308961c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:19.640344 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b985aab-8567-4f4a-8816-d26d8308961c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:07:19.645178 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-5b985aab-8567-4f4a-8816-d26d8308961c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 17:07:19.645178 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:07:19.645178 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:07:19.645178 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:07:19.645178 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:07:19.645178 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:07:19.645178 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:07:19.645178 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:07:19.645178 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:07:19.645178 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:07:19.645178 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:07:19.645178 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:07:19.645178 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:07:19.645178 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:07:19.645178 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:07:19.645178 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:07:19.646005 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:07:19.646005 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:07:19.646005 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:07:19.646005 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:07:19.646005 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:07:19.646005 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:07:19.646005 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:07:19.646005 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:07:19.646005 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:07:19.646005 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:07:19.646005 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:07:19.646005 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application with Lock( Aug 24 17:07:19.646005 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:07:19.646005 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:07:19.646005 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:07:19.646005 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:07:19.646761 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:07:19.646761 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:07:19.646761 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:07:19.646761 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:07:19.646761 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:07:19.646761 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:07:19.646761 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:07:19.646761 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:07:19.646761 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:07:19.646761 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:07:19.646761 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 17:07:19.646761 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:07:19.646761 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 76/150] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 24 17:07:19 2022] GET /identity/v3/roles/reader => generated 84 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:07:19.647461 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b985aab-8567-4f4a-8816-d26d8308961c None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:19.760403 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:19.772987 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a90e3aa-7be7-4ca6-a570-7d66ac3dbf51 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:19.791734 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b985aab-8567-4f4a-8816-d26d8308961c None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:19.843896 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17c9e6e-ea61-422f-a502-e2bbf1c3a42f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:19.844528 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17c9e6e-ea61-422f-a502-e2bbf1c3a42f None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:19.844815 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17c9e6e-ea61-422f-a502-e2bbf1c3a42f None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:19.845094 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17c9e6e-ea61-422f-a502-e2bbf1c3a42f None admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:19.845779 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17c9e6e-ea61-422f-a502-e2bbf1c3a42f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:07:19.846085 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17c9e6e-ea61-422f-a502-e2bbf1c3a42f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:19.847516 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17c9e6e-ea61-422f-a502-e2bbf1c3a42f None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:19.861278 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 77/151] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 24 17:07:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:19.861454 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132585ba-c5ac-401f-91cb-8e861443110a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:19.861992 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132585ba-c5ac-401f-91cb-8e861443110a None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:19.862163 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17c9e6e-ea61-422f-a502-e2bbf1c3a42f None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:19.862327 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132585ba-c5ac-401f-91cb-8e861443110a None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:19.862550 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132585ba-c5ac-401f-91cb-8e861443110a None admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:19.863265 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132585ba-c5ac-401f-91cb-8e861443110a None admin] RBAC: Adding query filter params (name=member) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:07:19.863559 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132585ba-c5ac-401f-91cb-8e861443110a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:19.865072 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132585ba-c5ac-401f-91cb-8e861443110a None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:19.882357 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 75/152] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Aug 24 17:07:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 411 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:19.883251 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132585ba-c5ac-401f-91cb-8e861443110a None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:19.939277 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:20.075953 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132585ba-c5ac-401f-91cb-8e861443110a None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:20.078195 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17c9e6e-ea61-422f-a502-e2bbf1c3a42f None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:20.141177 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf6749d-827b-4e25-adc7-8f6b18e959ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:20.142662 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf6749d-827b-4e25-adc7-8f6b18e959ec None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:20.143630 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf6749d-827b-4e25-adc7-8f6b18e959ec None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:20.144559 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf6749d-827b-4e25-adc7-8f6b18e959ec None admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:20.146206 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf6749d-827b-4e25-adc7-8f6b18e959ec None admin] RBAC: Adding query filter params (name=reader) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:07:20.147358 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf6749d-827b-4e25-adc7-8f6b18e959ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:20.148155 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa52040-f35a-4f1b-95f6-055b39c6dee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:20.149758 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf6749d-827b-4e25-adc7-8f6b18e959ec None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:20.150595 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa52040-f35a-4f1b-95f6-055b39c6dee3 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:20.151096 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa52040-f35a-4f1b-95f6-055b39c6dee3 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:20.151582 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa52040-f35a-4f1b-95f6-055b39c6dee3 None admin] PATH_INFO: `/v3/users/20b73f4d8b424258a870e87b04c9a532` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:20.155530 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa52040-f35a-4f1b-95f6-055b39c6dee3 None admin] RBAC: Authorizing `identity:get_user(user_id=20b73f4d8b424258a870e87b04c9a532)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:20.157165 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa52040-f35a-4f1b-95f6-055b39c6dee3 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:20.161772 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 76/153] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 24 17:07:19 2022] GET /identity/v3/users/20b73f4d8b424258a870e87b04c9a532 => generated 247 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:20.162913 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa52040-f35a-4f1b-95f6-055b39c6dee3 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:20.163212 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:20.169206 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 78/154] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Aug 24 17:07:19 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:20.171451 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4213a90d-f769-4db2-8ed2-fe86cb25f210 None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:20.172937 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4213a90d-f769-4db2-8ed2-fe86cb25f210 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:20.173407 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4213a90d-f769-4db2-8ed2-fe86cb25f210 None None] PATH_INFO: `/` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:20.179733 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 79/155] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:07:20 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:07:20.180959 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d105e3f-c7b8-4f21-a0cf-42794a80be89 None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:20.181645 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d105e3f-c7b8-4f21-a0cf-42794a80be89 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:20.187281 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d105e3f-c7b8-4f21-a0cf-42794a80be89 None None] PATH_INFO: `/` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:20.188604 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 80/156] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:07:20 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:07:20.193218 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d105e3f-c7b8-4f21-a0cf-42794a80be89 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:20.331308 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:20.343416 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa52040-f35a-4f1b-95f6-055b39c6dee3 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:20.355931 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d105e3f-c7b8-4f21-a0cf-42794a80be89 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:20.418219 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fd97e1-fc03-4f4c-af0e-7de87aa007fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:20.420084 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fd97e1-fc03-4f4c-af0e-7de87aa007fe None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:20.421011 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fd97e1-fc03-4f4c-af0e-7de87aa007fe None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:20.421794 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fd97e1-fc03-4f4c-af0e-7de87aa007fe None admin] PATH_INFO: `/v3/users/bd42523e33c34e618cc8f1f75df2b677` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:20.423687 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9e1056-90d4-4895-b16c-960b8996873e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:20.424541 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fd97e1-fc03-4f4c-af0e-7de87aa007fe None admin] RBAC: Authorizing `identity:get_user(user_id=bd42523e33c34e618cc8f1f75df2b677)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:20.426816 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fd97e1-fc03-4f4c-af0e-7de87aa007fe None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:20.427095 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9e1056-90d4-4895-b16c-960b8996873e None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:20.427562 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9e1056-90d4-4895-b16c-960b8996873e None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:20.428097 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9e1056-90d4-4895-b16c-960b8996873e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:20.432884 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 77/157] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 24 17:07:20 2022] GET /identity/v3/users/bd42523e33c34e618cc8f1f75df2b677 => generated 275 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:20.433928 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fd97e1-fc03-4f4c-af0e-7de87aa007fe None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:20.504686 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:20.508156 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9e1056-90d4-4895-b16c-960b8996873e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:20.509965 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9e1056-90d4-4895-b16c-960b8996873e None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:20.629968 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 81/158] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 24 17:07:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 17:07:20.631927 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fd97e1-fc03-4f4c-af0e-7de87aa007fe None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:20.635160 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab36c443-4e23-483b-b142-44be22b777a2 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:20.635860 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab36c443-4e23-483b-b142-44be22b777a2 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:20.637147 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab36c443-4e23-483b-b142-44be22b777a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:20.674341 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-ab36c443-4e23-483b-b142-44be22b777a2 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:07:20.676933 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-ab36c443-4e23-483b-b142-44be22b777a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:07:20.698518 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d16730-6d42-4555-bc0f-18cc5b2db800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:20.699510 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d16730-6d42-4555-bc0f-18cc5b2db800 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:20.700634 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d16730-6d42-4555-bc0f-18cc5b2db800 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:20.702099 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d16730-6d42-4555-bc0f-18cc5b2db800 None admin] PATH_INFO: `/v3/users/8fbef5751d094b0abe9e4ca3657fea9e` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:20.705900 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d16730-6d42-4555-bc0f-18cc5b2db800 None admin] RBAC: Authorizing `identity:get_user(user_id=8fbef5751d094b0abe9e4ca3657fea9e)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:20.707694 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d16730-6d42-4555-bc0f-18cc5b2db800 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:20.711120 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 78/159] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 24 17:07:20 2022] GET /identity/v3/users/8fbef5751d094b0abe9e4ca3657fea9e => generated 289 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:20.716174 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1a1717-859f-431f-b9b6-8d6088d6179c None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:20.718868 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1a1717-859f-431f-b9b6-8d6088d6179c None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:20.719430 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1a1717-859f-431f-b9b6-8d6088d6179c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:20.759686 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-2e1a1717-859f-431f-b9b6-8d6088d6179c None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:07:20.761236 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-2e1a1717-859f-431f-b9b6-8d6088d6179c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:07:20.809540 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:20.817405 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 82/160] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:07:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:07:20.820661 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab36c443-4e23-483b-b142-44be22b777a2 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:20.854681 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:20.884416 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 79/161] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:07:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:07:20.885482 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1a1717-859f-431f-b9b6-8d6088d6179c None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:20.956750 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab36c443-4e23-483b-b142-44be22b777a2 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:20.988698 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1a1717-859f-431f-b9b6-8d6088d6179c None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:21.009892 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06726c20-7408-4fb5-afa2-5ffa9b49d045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:21.010602 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06726c20-7408-4fb5-afa2-5ffa9b49d045 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:21.011075 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06726c20-7408-4fb5-afa2-5ffa9b49d045 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:21.011564 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06726c20-7408-4fb5-afa2-5ffa9b49d045 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:21.027693 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c906b6a-fe73-4d8c-899c-a1a08a7b7c33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:21.028678 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c906b6a-fe73-4d8c-899c-a1a08a7b7c33 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:21.029157 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c906b6a-fe73-4d8c-899c-a1a08a7b7c33 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:21.029694 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c906b6a-fe73-4d8c-899c-a1a08a7b7c33 None admin] PATH_INFO: `/v3/projects/9161a24523d545a8939fb56c3c930122` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:21.031089 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c906b6a-fe73-4d8c-899c-a1a08a7b7c33 None admin] RBAC: Authorizing `identity:get_project(project_id=9161a24523d545a8939fb56c3c930122)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:21.033057 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c906b6a-fe73-4d8c-899c-a1a08a7b7c33 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:21.034937 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 80/162] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 24 17:07:20 2022] GET /identity/v3/projects/9161a24523d545a8939fb56c3c930122 => generated 298 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:21.035902 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c906b6a-fe73-4d8c-899c-a1a08a7b7c33 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:21.064175 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:21.133391 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06726c20-7408-4fb5-afa2-5ffa9b49d045 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:21.135313 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06726c20-7408-4fb5-afa2-5ffa9b49d045 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:21.170939 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c906b6a-fe73-4d8c-899c-a1a08a7b7c33 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:21.221307 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:21.267060 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26180bf7-1abe-4150-a88d-dfcb87ce5975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:21.268006 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26180bf7-1abe-4150-a88d-dfcb87ce5975 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:21.268006 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26180bf7-1abe-4150-a88d-dfcb87ce5975 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:21.268270 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26180bf7-1abe-4150-a88d-dfcb87ce5975 None admin] PATH_INFO: `/v3/projects/9161a24523d545a8939fb56c3c930122` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:21.269461 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26180bf7-1abe-4150-a88d-dfcb87ce5975 None admin] RBAC: Authorizing `identity:get_project(project_id=9161a24523d545a8939fb56c3c930122)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:21.270953 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26180bf7-1abe-4150-a88d-dfcb87ce5975 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:21.272901 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 81/163] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 24 17:07:21 2022] GET /identity/v3/projects/9161a24523d545a8939fb56c3c930122 => generated 298 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:21.273667 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26180bf7-1abe-4150-a88d-dfcb87ce5975 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:21.370881 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 83/164] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 24 17:07:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 551 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 17:07:21.372453 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4bc851-097f-4d78-9e24-a6fa9d01300a None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:21.372881 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4bc851-097f-4d78-9e24-a6fa9d01300a None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:21.373869 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4bc851-097f-4d78-9e24-a6fa9d01300a None None] PATH_INFO: `/` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:21.375551 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 84/165] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:07:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:07:21.376456 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4bc851-097f-4d78-9e24-a6fa9d01300a None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:21.387352 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:21.395868 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26180bf7-1abe-4150-a88d-dfcb87ce5975 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:21.433638 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b5847d-92fb-44e8-abbe-049f7d4d10b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:21.434617 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b5847d-92fb-44e8-abbe-049f7d4d10b9 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:21.435129 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b5847d-92fb-44e8-abbe-049f7d4d10b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:21.435658 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b5847d-92fb-44e8-abbe-049f7d4d10b9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:21.437083 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b5847d-92fb-44e8-abbe-049f7d4d10b9 None admin] RBAC: Adding query filter params (role.id=c5346bc54abf445f9900edf94264c850, scope.project.id=9161a24523d545a8939fb56c3c930122, user.id=20b73f4d8b424258a870e87b04c9a532) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:07:21.437624 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b5847d-92fb-44e8-abbe-049f7d4d10b9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:21.439297 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b5847d-92fb-44e8-abbe-049f7d4d10b9 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:21.445667 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 82/166] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 24 17:07:21 2022] GET /identity/v3/role_assignments?user.id=20b73f4d8b424258a870e87b04c9a532&scope.project.id=9161a24523d545a8939fb56c3c930122&role.id=c5346bc54abf445f9900edf94264c850 => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:21.446715 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b5847d-92fb-44e8-abbe-049f7d4d10b9 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:21.543780 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4bc851-097f-4d78-9e24-a6fa9d01300a None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:21.606236 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:21.613852 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6ab4c1-fb37-4c87-8abd-04c5d270d39f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:21.614824 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6ab4c1-fb37-4c87-8abd-04c5d270d39f None admin] REQUEST_METHOD: `PUT` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:21.615454 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6ab4c1-fb37-4c87-8abd-04c5d270d39f None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:21.615879 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6ab4c1-fb37-4c87-8abd-04c5d270d39f None admin] PATH_INFO: `/v3/projects/9161a24523d545a8939fb56c3c930122/users/8fbef5751d094b0abe9e4ca3657fea9e/roles/7f65138ab599491194e7f91aa424ceee` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:21.625754 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6ab4c1-fb37-4c87-8abd-04c5d270d39f None admin] RBAC: Authorizing `identity:create_grant(project_id=9161a24523d545a8939fb56c3c930122, user_id=8fbef5751d094b0abe9e4ca3657fea9e, role_id=7f65138ab599491194e7f91aa424ceee)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:21.625754 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6ab4c1-fb37-4c87-8abd-04c5d270d39f None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:21.640128 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b5847d-92fb-44e8-abbe-049f7d4d10b9 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:21.677890 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 85/167] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Aug 24 17:07:21 2022] PUT /identity/v3/projects/9161a24523d545a8939fb56c3c930122/users/8fbef5751d094b0abe9e4ca3657fea9e/roles/7f65138ab599491194e7f91aa424ceee => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:07:21.684437 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6ab4c1-fb37-4c87-8abd-04c5d270d39f None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:21.684809 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb8e09a-f019-4c55-a82f-88a251134d75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:21.685667 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb8e09a-f019-4c55-a82f-88a251134d75 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:21.685952 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb8e09a-f019-4c55-a82f-88a251134d75 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:21.686275 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb8e09a-f019-4c55-a82f-88a251134d75 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:21.708438 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb8e09a-f019-4c55-a82f-88a251134d75 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:07:21.715347 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-0eb8e09a-f019-4c55-a82f-88a251134d75 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 17:07:21.715347 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:07:21.715347 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:07:21.715347 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:07:21.715347 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:07:21.715347 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:07:21.715347 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:07:21.715347 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:07:21.715347 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:07:21.715347 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:07:21.715347 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:07:21.715347 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:07:21.715347 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:07:21.715347 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:07:21.715347 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:07:21.715347 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:07:21.716483 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:07:21.716483 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:07:21.716483 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:07:21.716483 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:07:21.716483 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:07:21.716483 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:07:21.716483 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:07:21.716483 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:07:21.716483 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:07:21.716483 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:07:21.716483 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:07:21.716483 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 24 17:07:21.716483 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:07:21.716483 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:07:21.716483 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:07:21.716483 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:07:21.718245 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:07:21.718245 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:07:21.718245 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:07:21.718245 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:07:21.718245 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:07:21.718245 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:07:21.718245 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:07:21.718245 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:07:21.718245 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:07:21.718245 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:07:21.718245 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 17:07:21.718245 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:07:21.721392 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 83/168] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Aug 24 17:07:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:07:21.723084 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d266ecf4-c8e9-4ccb-a799-e688db5d8747 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:21.723566 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d266ecf4-c8e9-4ccb-a799-e688db5d8747 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:21.724081 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d266ecf4-c8e9-4ccb-a799-e688db5d8747 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:21.746273 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-d266ecf4-c8e9-4ccb-a799-e688db5d8747 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:07:21.747936 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-d266ecf4-c8e9-4ccb-a799-e688db5d8747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:07:21.764553 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:21.818412 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6ab4c1-fb37-4c87-8abd-04c5d270d39f None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:21.826117 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 84/169] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:07:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:07:21.827600 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d266ecf4-c8e9-4ccb-a799-e688db5d8747 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:21.856420 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d299002e-4b85-4fe9-bc81-ad2599bb41b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:21.858057 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d299002e-4b85-4fe9-bc81-ad2599bb41b7 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:21.858057 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d299002e-4b85-4fe9-bc81-ad2599bb41b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:21.858312 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d299002e-4b85-4fe9-bc81-ad2599bb41b7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:21.872949 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d299002e-4b85-4fe9-bc81-ad2599bb41b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:07:21.878777 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-d299002e-4b85-4fe9-bc81-ad2599bb41b7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:07:21.878777 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:07:21.878777 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:07:21.878777 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:07:21.878777 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:07:21.878777 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:07:21.878777 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:07:21.878777 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:07:21.878777 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:07:21.878777 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:07:21.878777 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:07:21.878777 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:07:21.878777 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:07:21.878777 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:07:21.878777 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:07:21.878777 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:07:21.879710 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:07:21.879710 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:07:21.879710 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:07:21.879710 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:07:21.879710 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:07:21.879710 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:07:21.879710 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:07:21.879710 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:07:21.879710 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:07:21.879710 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:07:21.879710 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:07:21.879710 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application with Lock( Aug 24 17:07:21.879710 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:07:21.879710 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:07:21.879710 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:07:21.879710 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:07:21.880603 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:07:21.880603 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:07:21.880603 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:07:21.880603 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:07:21.880603 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:07:21.880603 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:07:21.880603 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:07:21.880603 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:07:21.880603 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:07:21.880603 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:07:21.880603 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:07:21.880603 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:07:21.880603 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 86/170] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 24 17:07:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:07:21.881261 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080a2a42-15eb-4260-9bc7-a9bce74ec665 None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:21.882111 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080a2a42-15eb-4260-9bc7-a9bce74ec665 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:21.882111 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080a2a42-15eb-4260-9bc7-a9bce74ec665 None None] PATH_INFO: `/` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:21.884374 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 87/171] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:07:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:07:21.884374 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080a2a42-15eb-4260-9bc7-a9bce74ec665 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:21.955335 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:21.957776 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d266ecf4-c8e9-4ccb-a799-e688db5d8747 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:21.994051 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c24d94-ca74-4153-8558-2e68e597dcd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:21.994924 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c24d94-ca74-4153-8558-2e68e597dcd3 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:21.995213 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c24d94-ca74-4153-8558-2e68e597dcd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:21.995536 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c24d94-ca74-4153-8558-2e68e597dcd3 None admin] PATH_INFO: `/v3/projects/9161a24523d545a8939fb56c3c930122` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:21.996673 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c24d94-ca74-4153-8558-2e68e597dcd3 None admin] RBAC: Authorizing `identity:get_project(project_id=9161a24523d545a8939fb56c3c930122)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:21.998365 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c24d94-ca74-4153-8558-2e68e597dcd3 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:22.000575 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 85/172] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 24 17:07:21 2022] GET /identity/v3/projects/9161a24523d545a8939fb56c3c930122 => generated 298 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:22.001179 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c24d94-ca74-4153-8558-2e68e597dcd3 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:22.020406 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080a2a42-15eb-4260-9bc7-a9bce74ec665 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:22.036008 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:22.066401 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b7d52f-53ec-4d79-8a0c-0cbfd18785e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:22.067041 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b7d52f-53ec-4d79-8a0c-0cbfd18785e4 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:22.067592 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b7d52f-53ec-4d79-8a0c-0cbfd18785e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:22.067701 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b7d52f-53ec-4d79-8a0c-0cbfd18785e4 None admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:22.068338 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b7d52f-53ec-4d79-8a0c-0cbfd18785e4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:07:22.068713 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b7d52f-53ec-4d79-8a0c-0cbfd18785e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:22.070173 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b7d52f-53ec-4d79-8a0c-0cbfd18785e4 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:22.080104 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 88/173] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 24 17:07:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:22.081803 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab016d1-53be-4944-92ba-bc652589136f None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:22.081803 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab016d1-53be-4944-92ba-bc652589136f None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:22.082100 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab016d1-53be-4944-92ba-bc652589136f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:22.102798 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-eab016d1-53be-4944-92ba-bc652589136f None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:07:22.104136 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-eab016d1-53be-4944-92ba-bc652589136f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:07:22.140676 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c24d94-ca74-4153-8558-2e68e597dcd3 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:22.174462 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 89/174] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:07:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:07:22.178752 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5f6118-3585-4c99-86ab-d6acf8771acd None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:22.178752 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5f6118-3585-4c99-86ab-d6acf8771acd None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:22.178752 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5f6118-3585-4c99-86ab-d6acf8771acd None None] PATH_INFO: `/` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:22.178752 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 90/175] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:07:22 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:07:22.178752 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5f6118-3585-4c99-86ab-d6acf8771acd None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:22.184946 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc500c3f-65d7-45cb-8875-19efdc277c39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:22.185189 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc500c3f-65d7-45cb-8875-19efdc277c39 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:22.185433 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc500c3f-65d7-45cb-8875-19efdc277c39 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:22.185732 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc500c3f-65d7-45cb-8875-19efdc277c39 None admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:22.186456 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc500c3f-65d7-45cb-8875-19efdc277c39 None admin] RBAC: Adding query filter params (name=member) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:07:22.186757 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc500c3f-65d7-45cb-8875-19efdc277c39 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:22.188378 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc500c3f-65d7-45cb-8875-19efdc277c39 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:22.198275 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 86/176] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Aug 24 17:07:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:22.203849 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc500c3f-65d7-45cb-8875-19efdc277c39 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:22.228399 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:22.296690 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5f6118-3585-4c99-86ab-d6acf8771acd None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:22.301961 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:22.325293 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc500c3f-65d7-45cb-8875-19efdc277c39 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:22.340529 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1834bdc5-3efe-4c52-8f76-5f8a4ade4dfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:22.341139 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1834bdc5-3efe-4c52-8f76-5f8a4ade4dfc None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:22.341644 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1834bdc5-3efe-4c52-8f76-5f8a4ade4dfc None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:22.341972 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1834bdc5-3efe-4c52-8f76-5f8a4ade4dfc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:22.342708 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1834bdc5-3efe-4c52-8f76-5f8a4ade4dfc None admin] RBAC: Adding query filter params (role.id=60dc7e16aee746d5a3e8aa27d7d8525c, scope.project.id=9161a24523d545a8939fb56c3c930122, user.id=bd42523e33c34e618cc8f1f75df2b677) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:07:22.343515 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1834bdc5-3efe-4c52-8f76-5f8a4ade4dfc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:22.345275 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1834bdc5-3efe-4c52-8f76-5f8a4ade4dfc None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:22.353331 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 91/177] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 24 17:07:22 2022] GET /identity/v3/role_assignments?user.id=bd42523e33c34e618cc8f1f75df2b677&scope.project.id=9161a24523d545a8939fb56c3c930122&role.id=60dc7e16aee746d5a3e8aa27d7d8525c => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:22.354661 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1834bdc5-3efe-4c52-8f76-5f8a4ade4dfc None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:22.367750 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e839dac3-d88a-4459-8361-80e03d8df83c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:22.368773 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e839dac3-d88a-4459-8361-80e03d8df83c None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:22.369249 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e839dac3-d88a-4459-8361-80e03d8df83c None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:22.369715 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e839dac3-d88a-4459-8361-80e03d8df83c None admin] PATH_INFO: `/v3/users/bd42523e33c34e618cc8f1f75df2b677` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:22.371323 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e839dac3-d88a-4459-8361-80e03d8df83c None admin] RBAC: Authorizing `identity:get_user(user_id=bd42523e33c34e618cc8f1f75df2b677)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:22.373026 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e839dac3-d88a-4459-8361-80e03d8df83c None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:22.375803 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 87/178] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 24 17:07:22 2022] GET /identity/v3/users/bd42523e33c34e618cc8f1f75df2b677 => generated 275 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:22.377183 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d48d93f-c791-43c1-a6b7-c17e52493b5a None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:22.377655 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d48d93f-c791-43c1-a6b7-c17e52493b5a None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:22.378700 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d48d93f-c791-43c1-a6b7-c17e52493b5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:22.406417 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-6d48d93f-c791-43c1-a6b7-c17e52493b5a None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:07:22.407678 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-6d48d93f-c791-43c1-a6b7-c17e52493b5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:07:22.519913 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 88/179] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:07:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:07:22.520931 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d48d93f-c791-43c1-a6b7-c17e52493b5a None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:22.547275 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1834bdc5-3efe-4c52-8f76-5f8a4ade4dfc None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:22.567246 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:22.603072 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d1a2c8-19a5-43e0-9b60-aa1caa4f321c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:22.604103 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d1a2c8-19a5-43e0-9b60-aa1caa4f321c None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:22.604533 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d1a2c8-19a5-43e0-9b60-aa1caa4f321c None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:22.604977 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d1a2c8-19a5-43e0-9b60-aa1caa4f321c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:22.619563 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d1a2c8-19a5-43e0-9b60-aa1caa4f321c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:07:22.627203 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-91d1a2c8-19a5-43e0-9b60-aa1caa4f321c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:07:22.627203 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:07:22.627203 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:07:22.627203 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:07:22.627203 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:07:22.627203 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:07:22.627203 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:07:22.627203 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:07:22.627203 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:07:22.627203 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:07:22.627203 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:07:22.627203 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:07:22.627203 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:07:22.627203 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:07:22.627203 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:07:22.627203 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:07:22.628100 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:07:22.628100 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:07:22.628100 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:07:22.628100 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:07:22.628100 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:07:22.628100 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:07:22.628100 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:07:22.628100 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:07:22.628100 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:07:22.628100 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:07:22.628100 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:07:22.628100 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application with Lock( Aug 24 17:07:22.628100 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:07:22.628100 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:07:22.628100 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:07:22.628100 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:07:22.628879 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:07:22.628879 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:07:22.628879 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:07:22.628879 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:07:22.628879 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:07:22.628879 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:07:22.628879 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:07:22.628879 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:07:22.628879 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:07:22.628879 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:07:22.628879 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:07:22.628879 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:07:22.631344 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 92/180] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Aug 24 17:07:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:07:22.632514 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d1a2c8-19a5-43e0-9b60-aa1caa4f321c None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:22.639903 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:22.673330 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d48d93f-c791-43c1-a6b7-c17e52493b5a None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:22.717290 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5033fc2-6c1b-4d68-990d-efb8912b70ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:22.717952 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5033fc2-6c1b-4d68-990d-efb8912b70ef None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:22.719033 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5033fc2-6c1b-4d68-990d-efb8912b70ef None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:22.723086 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5033fc2-6c1b-4d68-990d-efb8912b70ef None admin] PATH_INFO: `/v3/users/bd42523e33c34e618cc8f1f75df2b677` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:22.724939 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5033fc2-6c1b-4d68-990d-efb8912b70ef None admin] RBAC: Authorizing `identity:get_user(user_id=bd42523e33c34e618cc8f1f75df2b677)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:22.727073 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5033fc2-6c1b-4d68-990d-efb8912b70ef None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:22.729565 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 89/181] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 24 17:07:22 2022] GET /identity/v3/users/bd42523e33c34e618cc8f1f75df2b677 => generated 275 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:22.731266 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5033fc2-6c1b-4d68-990d-efb8912b70ef None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:22.756348 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d1a2c8-19a5-43e0-9b60-aa1caa4f321c None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:22.801093 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc913693-1d6f-4204-9811-9aca14d17797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:22.801886 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc913693-1d6f-4204-9811-9aca14d17797 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:22.802191 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc913693-1d6f-4204-9811-9aca14d17797 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:22.802857 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc913693-1d6f-4204-9811-9aca14d17797 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:22.851096 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5033fc2-6c1b-4d68-990d-efb8912b70ef None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:22.851289 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:22.888378 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e875c1-2c63-4f23-aa6a-938c7b297681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:22.889037 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e875c1-2c63-4f23-aa6a-938c7b297681 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:22.889317 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e875c1-2c63-4f23-aa6a-938c7b297681 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:22.889592 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e875c1-2c63-4f23-aa6a-938c7b297681 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:22.890250 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc913693-1d6f-4204-9811-9aca14d17797 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:22.891734 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc913693-1d6f-4204-9811-9aca14d17797 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:22.900767 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e875c1-2c63-4f23-aa6a-938c7b297681 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:07:22.905510 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-50e875c1-2c63-4f23-aa6a-938c7b297681 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 17:07:22.905510 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:07:22.905510 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:07:22.905510 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:07:22.905510 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:07:22.905510 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:07:22.905510 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:07:22.905510 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:07:22.905510 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:07:22.905510 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:07:22.905510 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:07:22.905510 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:07:22.905510 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:07:22.905510 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:07:22.905510 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:07:22.905510 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:07:22.906430 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:07:22.906430 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:07:22.906430 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:07:22.906430 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:07:22.906430 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:07:22.906430 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:07:22.906430 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:07:22.906430 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:07:22.906430 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:07:22.906430 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:07:22.906430 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:07:22.906430 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 24 17:07:22.906430 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:07:22.906430 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:07:22.906430 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:07:22.906430 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:07:22.907942 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:07:22.907942 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:07:22.907942 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:07:22.907942 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:07:22.907942 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:07:22.907942 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:07:22.907942 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:07:22.907942 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:07:22.907942 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:07:22.907942 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:07:22.907942 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 17:07:22.907942 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:07:22.907942 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 90/182] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Aug 24 17:07:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:07:22.907942 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e875c1-2c63-4f23-aa6a-938c7b297681 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:22.912772 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:23.011368 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 93/183] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 24 17:07:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 17:07:23.012584 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc913693-1d6f-4204-9811-9aca14d17797 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:23.022055 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e875c1-2c63-4f23-aa6a-938c7b297681 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:23.058285 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ddf3b9-cd8b-41b2-9eb6-b4ae1d8564fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:23.058815 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ddf3b9-cd8b-41b2-9eb6-b4ae1d8564fe None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:23.059124 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ddf3b9-cd8b-41b2-9eb6-b4ae1d8564fe None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:23.059397 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ddf3b9-cd8b-41b2-9eb6-b4ae1d8564fe None admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:23.060061 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ddf3b9-cd8b-41b2-9eb6-b4ae1d8564fe None admin] RBAC: Adding query filter params (name=admin) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:07:23.060362 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ddf3b9-cd8b-41b2-9eb6-b4ae1d8564fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:23.061764 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ddf3b9-cd8b-41b2-9eb6-b4ae1d8564fe None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:23.070281 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 91/184] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 24 17:07:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:23.072057 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ddf3b9-cd8b-41b2-9eb6-b4ae1d8564fe None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:23.080146 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:23.119652 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc913693-1d6f-4204-9811-9aca14d17797 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:23.144698 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:23.154002 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869eb24a-dab8-4db2-922e-f6cd11c5dba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:23.155158 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869eb24a-dab8-4db2-922e-f6cd11c5dba3 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:23.155439 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869eb24a-dab8-4db2-922e-f6cd11c5dba3 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:23.155706 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869eb24a-dab8-4db2-922e-f6cd11c5dba3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:23.183679 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ddf3b9-cd8b-41b2-9eb6-b4ae1d8564fe None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:23.220563 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894f0474-1f62-4234-ac6e-2cddb53ccddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:23.221193 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894f0474-1f62-4234-ac6e-2cddb53ccddd None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:23.221478 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894f0474-1f62-4234-ac6e-2cddb53ccddd None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:23.221749 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894f0474-1f62-4234-ac6e-2cddb53ccddd None admin] PATH_INFO: `/v3/domains/default` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:23.222125 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869eb24a-dab8-4db2-922e-f6cd11c5dba3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:23.222996 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894f0474-1f62-4234-ac6e-2cddb53ccddd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:23.223734 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869eb24a-dab8-4db2-922e-f6cd11c5dba3 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:23.224496 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894f0474-1f62-4234-ac6e-2cddb53ccddd None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:23.226299 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 92/185] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 24 17:07:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:23.227115 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894f0474-1f62-4234-ac6e-2cddb53ccddd None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:23.293693 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:23.336152 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 94/186] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 24 17:07:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 17:07:23.336152 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869eb24a-dab8-4db2-922e-f6cd11c5dba3 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:23.340100 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894f0474-1f62-4234-ac6e-2cddb53ccddd None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:23.379498 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6597098f-b937-46bf-96ec-32cf66821e3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:23.380446 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6597098f-b937-46bf-96ec-32cf66821e3e None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:23.380800 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6597098f-b937-46bf-96ec-32cf66821e3e None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:23.381240 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6597098f-b937-46bf-96ec-32cf66821e3e None admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:23.381915 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6597098f-b937-46bf-96ec-32cf66821e3e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:07:23.382217 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6597098f-b937-46bf-96ec-32cf66821e3e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:23.383662 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6597098f-b937-46bf-96ec-32cf66821e3e None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:23.385252 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:23.393673 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 93/187] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 24 17:07:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:23.399347 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6597098f-b937-46bf-96ec-32cf66821e3e None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:23.440601 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869eb24a-dab8-4db2-922e-f6cd11c5dba3 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:23.483643 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cb525b-1ec9-4add-9ef7-afe0ef022222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:23.484284 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cb525b-1ec9-4add-9ef7-afe0ef022222 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:23.484565 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cb525b-1ec9-4add-9ef7-afe0ef022222 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:23.484849 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cb525b-1ec9-4add-9ef7-afe0ef022222 None admin] PATH_INFO: `/v3/projects/9161a24523d545a8939fb56c3c930122` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:23.485969 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cb525b-1ec9-4add-9ef7-afe0ef022222 None admin] RBAC: Authorizing `identity:get_project(project_id=9161a24523d545a8939fb56c3c930122)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:23.487433 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cb525b-1ec9-4add-9ef7-afe0ef022222 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:23.489257 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 95/188] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 24 17:07:23 2022] GET /identity/v3/projects/9161a24523d545a8939fb56c3c930122 => generated 298 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:23.489915 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cb525b-1ec9-4add-9ef7-afe0ef022222 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:23.510199 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6597098f-b937-46bf-96ec-32cf66821e3e None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:23.524753 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:23.547217 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e8903f-d68a-4fd5-8e4b-2d2f4b30edb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:23.547861 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e8903f-d68a-4fd5-8e4b-2d2f4b30edb6 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:23.551978 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e8903f-d68a-4fd5-8e4b-2d2f4b30edb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:23.551978 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e8903f-d68a-4fd5-8e4b-2d2f4b30edb6 None admin] PATH_INFO: `/v3/users/815d5ea515c943b3ae9100dfc2acc10a` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:23.565753 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e8903f-d68a-4fd5-8e4b-2d2f4b30edb6 None admin] RBAC: Authorizing `identity:get_user(user_id=815d5ea515c943b3ae9100dfc2acc10a)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:23.567251 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e8903f-d68a-4fd5-8e4b-2d2f4b30edb6 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:23.569238 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 94/189] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 24 17:07:23 2022] GET /identity/v3/users/815d5ea515c943b3ae9100dfc2acc10a => generated 283 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:23.569866 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e8903f-d68a-4fd5-8e4b-2d2f4b30edb6 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:23.609524 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cb525b-1ec9-4add-9ef7-afe0ef022222 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:23.641872 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:23.654773 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbaaa5b-4d58-41cf-923b-2b4b81eef607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:23.662805 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbaaa5b-4d58-41cf-923b-2b4b81eef607 None admin] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:23.662805 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbaaa5b-4d58-41cf-923b-2b4b81eef607 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:23.662805 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbaaa5b-4d58-41cf-923b-2b4b81eef607 None admin] PATH_INFO: `/v3/users` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:23.662805 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbaaa5b-4d58-41cf-923b-2b4b81eef607 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:23.662805 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbaaa5b-4d58-41cf-923b-2b4b81eef607 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:23.688682 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e8903f-d68a-4fd5-8e4b-2d2f4b30edb6 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:23.722936 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbbd3ae-e8fa-4535-a848-745a5abaef3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:23.723581 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbbd3ae-e8fa-4535-a848-745a5abaef3c None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:23.723863 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbbd3ae-e8fa-4535-a848-745a5abaef3c None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:23.724136 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbbd3ae-e8fa-4535-a848-745a5abaef3c None admin] PATH_INFO: `/v3/projects/d00aeeaba4ee4ac696756e040b6b0585` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:23.725260 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbbd3ae-e8fa-4535-a848-745a5abaef3c None admin] RBAC: Authorizing `identity:get_project(project_id=d00aeeaba4ee4ac696756e040b6b0585)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:23.726830 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbbd3ae-e8fa-4535-a848-745a5abaef3c None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:23.728509 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 95/190] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 24 17:07:23 2022] GET /identity/v3/projects/d00aeeaba4ee4ac696756e040b6b0585 => generated 312 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:23.729266 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbbd3ae-e8fa-4535-a848-745a5abaef3c None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:23.751127 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 96/191] 10.222.0.18 () {60 vars in 1132 bytes} [Wed Aug 24 17:07:23 2022] POST /identity/v3/users => generated 297 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:07:23.752344 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbaaa5b-4d58-41cf-923b-2b4b81eef607 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:23.857116 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbbd3ae-e8fa-4535-a848-745a5abaef3c None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:23.864363 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:23.874914 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbaaa5b-4d58-41cf-923b-2b4b81eef607 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:23.885412 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:23.898181 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb60000e-cd52-4b94-b226-0a49fd0dc459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:23.898825 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb60000e-cd52-4b94-b226-0a49fd0dc459 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:23.899134 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb60000e-cd52-4b94-b226-0a49fd0dc459 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:23.899407 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb60000e-cd52-4b94-b226-0a49fd0dc459 None admin] PATH_INFO: `/v3/users/815d5ea515c943b3ae9100dfc2acc10a` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:23.900549 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb60000e-cd52-4b94-b226-0a49fd0dc459 None admin] RBAC: Authorizing `identity:get_user(user_id=815d5ea515c943b3ae9100dfc2acc10a)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:23.901994 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb60000e-cd52-4b94-b226-0a49fd0dc459 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:23.903612 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 96/192] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 24 17:07:23 2022] GET /identity/v3/users/815d5ea515c943b3ae9100dfc2acc10a => generated 283 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:23.908788 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb60000e-cd52-4b94-b226-0a49fd0dc459 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:23.921517 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53e10e6-a939-43bc-9ce9-4b45927f9ff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:23.922422 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53e10e6-a939-43bc-9ce9-4b45927f9ff2 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:23.922700 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53e10e6-a939-43bc-9ce9-4b45927f9ff2 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:23.923000 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53e10e6-a939-43bc-9ce9-4b45927f9ff2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:23.923785 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53e10e6-a939-43bc-9ce9-4b45927f9ff2 None admin] RBAC: Adding query filter params (role.id=af19a7a2fae5422da5bd7daea4bbfccf, scope.project.id=9161a24523d545a8939fb56c3c930122, user.id=bd42523e33c34e618cc8f1f75df2b677) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:07:23.924043 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53e10e6-a939-43bc-9ce9-4b45927f9ff2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:23.925720 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53e10e6-a939-43bc-9ce9-4b45927f9ff2 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:23.932306 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 97/193] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 24 17:07:23 2022] GET /identity/v3/role_assignments?user.id=bd42523e33c34e618cc8f1f75df2b677&scope.project.id=9161a24523d545a8939fb56c3c930122&role.id=af19a7a2fae5422da5bd7daea4bbfccf => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:23.933130 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53e10e6-a939-43bc-9ce9-4b45927f9ff2 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:24.055952 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb60000e-cd52-4b94-b226-0a49fd0dc459 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:24.081917 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53e10e6-a939-43bc-9ce9-4b45927f9ff2 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:24.109663 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95f97aa-ac62-4dbf-b45d-3b71a1dc52d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:24.111027 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95f97aa-ac62-4dbf-b45d-3b71a1dc52d4 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:24.111784 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95f97aa-ac62-4dbf-b45d-3b71a1dc52d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:24.112263 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95f97aa-ac62-4dbf-b45d-3b71a1dc52d4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:24.122942 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de91e52-83d4-4d0d-86f4-352f99c7fe16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:24.123568 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de91e52-83d4-4d0d-86f4-352f99c7fe16 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:24.123849 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de91e52-83d4-4d0d-86f4-352f99c7fe16 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:24.124139 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de91e52-83d4-4d0d-86f4-352f99c7fe16 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:24.124875 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de91e52-83d4-4d0d-86f4-352f99c7fe16 None admin] RBAC: Adding query filter params (role.id=60dc7e16aee746d5a3e8aa27d7d8525c, scope.project.id=d00aeeaba4ee4ac696756e040b6b0585, user.id=bd42523e33c34e618cc8f1f75df2b677) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:07:24.125809 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de91e52-83d4-4d0d-86f4-352f99c7fe16 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:24.127668 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de91e52-83d4-4d0d-86f4-352f99c7fe16 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:24.135067 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 98/194] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 24 17:07:23 2022] GET /identity/v3/role_assignments?user.id=bd42523e33c34e618cc8f1f75df2b677&scope.project.id=d00aeeaba4ee4ac696756e040b6b0585&role.id=60dc7e16aee746d5a3e8aa27d7d8525c => generated 608 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:24.136357 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77197d15-cefd-49f1-b153-a9e43cc196b1 None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:24.136641 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77197d15-cefd-49f1-b153-a9e43cc196b1 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:24.137231 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77197d15-cefd-49f1-b153-a9e43cc196b1 None None] PATH_INFO: `/` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:24.138476 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 99/195] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:07:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:07:24.139794 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77197d15-cefd-49f1-b153-a9e43cc196b1 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:24.175889 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:24.197799 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:24.213056 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95f97aa-ac62-4dbf-b45d-3b71a1dc52d4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:24.214752 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95f97aa-ac62-4dbf-b45d-3b71a1dc52d4 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:24.303152 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77197d15-cefd-49f1-b153-a9e43cc196b1 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:24.348432 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5479a0-cbc2-4ce1-a8f7-5a171964e060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:24.349448 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5479a0-cbc2-4ce1-a8f7-5a171964e060 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:24.349928 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5479a0-cbc2-4ce1-a8f7-5a171964e060 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:24.350416 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5479a0-cbc2-4ce1-a8f7-5a171964e060 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:24.350521 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 97/196] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 24 17:07:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 17:07:24.354269 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90441f1f-d0a1-469e-9584-0a75d1ba17eb None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:24.354269 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90441f1f-d0a1-469e-9584-0a75d1ba17eb None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:24.354269 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90441f1f-d0a1-469e-9584-0a75d1ba17eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:24.388756 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-90441f1f-d0a1-469e-9584-0a75d1ba17eb None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:07:24.389772 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-90441f1f-d0a1-469e-9584-0a75d1ba17eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:07:24.459665 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5479a0-cbc2-4ce1-a8f7-5a171964e060 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:24.461605 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5479a0-cbc2-4ce1-a8f7-5a171964e060 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:24.476567 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 98/197] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:07:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:07:24.477222 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90441f1f-d0a1-469e-9584-0a75d1ba17eb None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:24.482882 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:24.511175 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:24.614264 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90441f1f-d0a1-469e-9584-0a75d1ba17eb None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:24.630781 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 100/198] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 24 17:07:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 491 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 17:07:24.637927 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5479a0-cbc2-4ce1-a8f7-5a171964e060 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:24.678721 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316d3b43-d116-42f9-bf3d-3d6cb5087b8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:24.683039 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316d3b43-d116-42f9-bf3d-3d6cb5087b8b None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:24.683039 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316d3b43-d116-42f9-bf3d-3d6cb5087b8b None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:24.683039 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316d3b43-d116-42f9-bf3d-3d6cb5087b8b None admin] PATH_INFO: `/v3/projects/94586413811643bb999468e84b659403` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:24.689294 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316d3b43-d116-42f9-bf3d-3d6cb5087b8b None admin] RBAC: Authorizing `identity:get_project(project_id=94586413811643bb999468e84b659403)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:24.689847 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316d3b43-d116-42f9-bf3d-3d6cb5087b8b None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:24.692875 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 99/199] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 24 17:07:24 2022] GET /identity/v3/projects/94586413811643bb999468e84b659403 => generated 302 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:24.702933 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316d3b43-d116-42f9-bf3d-3d6cb5087b8b None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:24.836546 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5479a0-cbc2-4ce1-a8f7-5a171964e060 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:24.864750 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:24.867376 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:24.879565 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316d3b43-d116-42f9-bf3d-3d6cb5087b8b None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:24.882821 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e881e8a4-f2ed-4bc6-8fe4-3fd86de79ae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:24.883502 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e881e8a4-f2ed-4bc6-8fe4-3fd86de79ae3 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:24.883780 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e881e8a4-f2ed-4bc6-8fe4-3fd86de79ae3 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:24.884056 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e881e8a4-f2ed-4bc6-8fe4-3fd86de79ae3 None admin] PATH_INFO: `/v3/projects/94586413811643bb999468e84b659403` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:24.885180 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e881e8a4-f2ed-4bc6-8fe4-3fd86de79ae3 None admin] RBAC: Authorizing `identity:get_project(project_id=94586413811643bb999468e84b659403)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:24.886612 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e881e8a4-f2ed-4bc6-8fe4-3fd86de79ae3 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:24.889130 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 101/200] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 24 17:07:24 2022] GET /identity/v3/projects/94586413811643bb999468e84b659403 => generated 302 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:07:24.889949 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e881e8a4-f2ed-4bc6-8fe4-3fd86de79ae3 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:24.919260 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a513aee2-4b0f-4b6f-a411-19a16b46e56e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:24.919884 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a513aee2-4b0f-4b6f-a411-19a16b46e56e None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:24.920170 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a513aee2-4b0f-4b6f-a411-19a16b46e56e None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:24.920441 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a513aee2-4b0f-4b6f-a411-19a16b46e56e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:24.921172 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a513aee2-4b0f-4b6f-a411-19a16b46e56e None admin] RBAC: Adding query filter params (role.id=c5346bc54abf445f9900edf94264c850, scope.project.id=94586413811643bb999468e84b659403, user.id=815d5ea515c943b3ae9100dfc2acc10a) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:07:24.921477 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a513aee2-4b0f-4b6f-a411-19a16b46e56e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:24.922928 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a513aee2-4b0f-4b6f-a411-19a16b46e56e None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:24.928266 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 100/201] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 24 17:07:24 2022] GET /identity/v3/role_assignments?user.id=815d5ea515c943b3ae9100dfc2acc10a&scope.project.id=94586413811643bb999468e84b659403&role.id=c5346bc54abf445f9900edf94264c850 => generated 259 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:24.933547 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a513aee2-4b0f-4b6f-a411-19a16b46e56e None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:25.002050 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e881e8a4-f2ed-4bc6-8fe4-3fd86de79ae3 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:25.041556 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d355ce97-8ed2-4cfe-8f2a-bf7bdcdbf78d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:25.041882 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a513aee2-4b0f-4b6f-a411-19a16b46e56e None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:25.042212 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d355ce97-8ed2-4cfe-8f2a-bf7bdcdbf78d None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:25.042523 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d355ce97-8ed2-4cfe-8f2a-bf7bdcdbf78d None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:25.042798 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d355ce97-8ed2-4cfe-8f2a-bf7bdcdbf78d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:25.055282 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d355ce97-8ed2-4cfe-8f2a-bf7bdcdbf78d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:07:25.061330 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-d355ce97-8ed2-4cfe-8f2a-bf7bdcdbf78d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 17:07:25.061330 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:07:25.061330 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:07:25.061330 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:07:25.061330 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:07:25.061330 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:07:25.061330 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:07:25.061330 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:07:25.061330 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:07:25.061330 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:07:25.061330 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:07:25.061330 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:07:25.061330 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:07:25.061330 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:07:25.061330 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:07:25.061330 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:07:25.062187 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:07:25.062187 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:07:25.062187 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:07:25.062187 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:07:25.062187 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:07:25.062187 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:07:25.062187 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:07:25.062187 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:07:25.062187 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:07:25.062187 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:07:25.062187 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:07:25.062187 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application with Lock( Aug 24 17:07:25.062187 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:07:25.062187 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:07:25.062187 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:07:25.062187 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:07:25.063093 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:07:25.063093 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:07:25.063093 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:07:25.063093 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:07:25.063093 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:07:25.063093 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:07:25.063093 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:07:25.063093 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:07:25.063093 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:07:25.063093 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:07:25.063093 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 17:07:25.063093 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:07:25.063093 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 102/202] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 24 17:07:24 2022] GET /identity/v3/roles/reader => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:07:25.069639 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d355ce97-8ed2-4cfe-8f2a-bf7bdcdbf78d None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:25.090278 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5517f57a-01e1-4ad2-b77b-714f4ea012bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:25.090931 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5517f57a-01e1-4ad2-b77b-714f4ea012bb None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:25.091316 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5517f57a-01e1-4ad2-b77b-714f4ea012bb None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:25.092744 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5517f57a-01e1-4ad2-b77b-714f4ea012bb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:25.093509 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5517f57a-01e1-4ad2-b77b-714f4ea012bb None admin] RBAC: Adding query filter params (role.id=af19a7a2fae5422da5bd7daea4bbfccf, scope.project.id=94586413811643bb999468e84b659403, user.id=815d5ea515c943b3ae9100dfc2acc10a) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:07:25.094103 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5517f57a-01e1-4ad2-b77b-714f4ea012bb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:25.095907 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5517f57a-01e1-4ad2-b77b-714f4ea012bb None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:25.103851 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 101/203] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 24 17:07:24 2022] GET /identity/v3/role_assignments?user.id=815d5ea515c943b3ae9100dfc2acc10a&scope.project.id=94586413811643bb999468e84b659403&role.id=af19a7a2fae5422da5bd7daea4bbfccf => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:25.156153 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:25.224300 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d355ce97-8ed2-4cfe-8f2a-bf7bdcdbf78d None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:25.263695 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d478fbe0-c7a6-414d-85d0-adae2b3249ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:25.264829 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d478fbe0-c7a6-414d-85d0-adae2b3249ad None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:25.265111 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d478fbe0-c7a6-414d-85d0-adae2b3249ad None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:25.265384 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d478fbe0-c7a6-414d-85d0-adae2b3249ad None admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:25.266060 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d478fbe0-c7a6-414d-85d0-adae2b3249ad None admin] RBAC: Adding query filter params (name=reader) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:07:25.266361 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d478fbe0-c7a6-414d-85d0-adae2b3249ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:25.267772 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d478fbe0-c7a6-414d-85d0-adae2b3249ad None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:25.276481 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 103/204] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Aug 24 17:07:25 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:25.282652 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5517f57a-01e1-4ad2-b77b-714f4ea012bb None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:25.307370 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:25.403693 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5517f57a-01e1-4ad2-b77b-714f4ea012bb None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:25.445068 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747c59bb-56af-4d63-adf8-be09e70da0a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:25.445733 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747c59bb-56af-4d63-adf8-be09e70da0a4 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:25.446016 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747c59bb-56af-4d63-adf8-be09e70da0a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:25.446294 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747c59bb-56af-4d63-adf8-be09e70da0a4 None admin] PATH_INFO: `/v3/users/8fbef5751d094b0abe9e4ca3657fea9e` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:25.447582 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747c59bb-56af-4d63-adf8-be09e70da0a4 None admin] RBAC: Authorizing `identity:get_user(user_id=8fbef5751d094b0abe9e4ca3657fea9e)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:25.449071 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747c59bb-56af-4d63-adf8-be09e70da0a4 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:25.450737 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 102/205] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 24 17:07:25 2022] GET /identity/v3/users/8fbef5751d094b0abe9e4ca3657fea9e => generated 289 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:25.457374 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d478fbe0-c7a6-414d-85d0-adae2b3249ad None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:25.608451 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d478fbe0-c7a6-414d-85d0-adae2b3249ad None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:25.622946 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:25.648782 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40162e8-ee6a-4509-b2ae-a195a2268a7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:25.649397 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40162e8-ee6a-4509-b2ae-a195a2268a7a None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:25.649679 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40162e8-ee6a-4509-b2ae-a195a2268a7a None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:25.649953 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40162e8-ee6a-4509-b2ae-a195a2268a7a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:25.719477 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40162e8-ee6a-4509-b2ae-a195a2268a7a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:25.720936 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40162e8-ee6a-4509-b2ae-a195a2268a7a None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:25.827738 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 104/206] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 24 17:07:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 17:07:25.833800 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747c59bb-56af-4d63-adf8-be09e70da0a4 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:25.976930 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:25.994726 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747c59bb-56af-4d63-adf8-be09e70da0a4 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:26.029991 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fea0fc-36d4-42ab-bfe7-9176270ad249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:26.030632 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fea0fc-36d4-42ab-bfe7-9176270ad249 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:26.030926 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fea0fc-36d4-42ab-bfe7-9176270ad249 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:26.031220 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fea0fc-36d4-42ab-bfe7-9176270ad249 None admin] PATH_INFO: `/v3/projects/9161a24523d545a8939fb56c3c930122` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:26.032355 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fea0fc-36d4-42ab-bfe7-9176270ad249 None admin] RBAC: Authorizing `identity:get_project(project_id=9161a24523d545a8939fb56c3c930122)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:26.033900 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fea0fc-36d4-42ab-bfe7-9176270ad249 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:26.040445 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 103/207] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 24 17:07:25 2022] GET /identity/v3/projects/9161a24523d545a8939fb56c3c930122 => generated 298 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:26.041846 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40162e8-ee6a-4509-b2ae-a195a2268a7a None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:26.063717 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:26.151338 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40162e8-ee6a-4509-b2ae-a195a2268a7a None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:26.151644 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8eea98-9224-46a0-b6f7-8f554419cf00 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:26.151833 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8eea98-9224-46a0-b6f7-8f554419cf00 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:26.152108 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8eea98-9224-46a0-b6f7-8f554419cf00 None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:26.153453 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 104/208] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:07:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 17:07:26.159191 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be96c482-75de-4bff-a68c-b469113e071e None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:26.159912 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be96c482-75de-4bff-a68c-b469113e071e None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:26.160185 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be96c482-75de-4bff-a68c-b469113e071e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:26.192791 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-be96c482-75de-4bff-a68c-b469113e071e None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:07:26.193977 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-be96c482-75de-4bff-a68c-b469113e071e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:07:26.204881 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17478f72-c8cc-4bbe-a732-0e26e68d29c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:26.205495 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17478f72-c8cc-4bbe-a732-0e26e68d29c9 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:26.205771 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17478f72-c8cc-4bbe-a732-0e26e68d29c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:26.206041 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17478f72-c8cc-4bbe-a732-0e26e68d29c9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:26.206766 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17478f72-c8cc-4bbe-a732-0e26e68d29c9 None admin] RBAC: Adding query filter params (role.id=7f65138ab599491194e7f91aa424ceee, scope.project.id=9161a24523d545a8939fb56c3c930122, user.id=8fbef5751d094b0abe9e4ca3657fea9e) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:07:26.207102 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17478f72-c8cc-4bbe-a732-0e26e68d29c9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:26.208530 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17478f72-c8cc-4bbe-a732-0e26e68d29c9 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:26.214563 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 105/209] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 24 17:07:26 2022] GET /identity/v3/role_assignments?user.id=8fbef5751d094b0abe9e4ca3657fea9e&scope.project.id=9161a24523d545a8939fb56c3c930122&role.id=7f65138ab599491194e7f91aa424ceee => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:07:26.307622 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 105/210] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:07:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 17:07:26.529950 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d0ccde-4695-4e9e-93c7-4a00aa82a549 None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:26.530248 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d0ccde-4695-4e9e-93c7-4a00aa82a549 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:26.530430 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d0ccde-4695-4e9e-93c7-4a00aa82a549 None None] PATH_INFO: `/` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:26.531824 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 106/211] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:07:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:07:26.538257 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604d5989-1894-400a-bf4f-4ca374199e32 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:26.538544 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604d5989-1894-400a-bf4f-4ca374199e32 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:26.538823 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604d5989-1894-400a-bf4f-4ca374199e32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:26.557682 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-604d5989-1894-400a-bf4f-4ca374199e32 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:07:26.558875 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-604d5989-1894-400a-bf4f-4ca374199e32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:07:26.630410 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 106/212] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:07:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:07:26.633314 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d0ccde-4695-4e9e-93c7-4a00aa82a549 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:26.713529 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:26.756489 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d0ccde-4695-4e9e-93c7-4a00aa82a549 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:26.791945 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4346a7-1925-46ea-8a34-c4adef5659f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:26.792502 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4346a7-1925-46ea-8a34-c4adef5659f6 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:26.792780 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4346a7-1925-46ea-8a34-c4adef5659f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:26.793149 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4346a7-1925-46ea-8a34-c4adef5659f6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:26.803620 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4346a7-1925-46ea-8a34-c4adef5659f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:07:26.808895 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-bd4346a7-1925-46ea-8a34-c4adef5659f6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:07:26.808895 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:07:26.808895 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:07:26.808895 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:07:26.808895 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:07:26.808895 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:07:26.808895 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:07:26.808895 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:07:26.808895 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:07:26.808895 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:07:26.808895 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:07:26.808895 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:07:26.808895 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:07:26.808895 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:07:26.808895 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:07:26.808895 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:07:26.809761 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:07:26.809761 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:07:26.809761 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:07:26.809761 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:07:26.809761 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:07:26.809761 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:07:26.809761 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:07:26.809761 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:07:26.809761 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:07:26.809761 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:07:26.809761 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:07:26.809761 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application with Lock( Aug 24 17:07:26.809761 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:07:26.809761 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:07:26.809761 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:07:26.809761 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:07:26.811113 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:07:26.811113 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:07:26.811113 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:07:26.811113 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:07:26.811113 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:07:26.811113 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:07:26.811113 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:07:26.811113 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:07:26.811113 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:07:26.811113 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:07:26.811113 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:07:26.811113 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:07:26.811113 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 107/213] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 24 17:07:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:07:26.816540 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604d5989-1894-400a-bf4f-4ca374199e32 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:26.843318 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:26.929365 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604d5989-1894-400a-bf4f-4ca374199e32 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:26.962255 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4346a7-1925-46ea-8a34-c4adef5659f6 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:26.966930 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c971a31d-cafb-4038-a883-e0ec188212c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:26.967565 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c971a31d-cafb-4038-a883-e0ec188212c0 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:26.967846 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c971a31d-cafb-4038-a883-e0ec188212c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:26.968120 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c971a31d-cafb-4038-a883-e0ec188212c0 None admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:26.968790 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c971a31d-cafb-4038-a883-e0ec188212c0 None admin] RBAC: Adding query filter params (name=member) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:07:26.969096 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c971a31d-cafb-4038-a883-e0ec188212c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:26.970487 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c971a31d-cafb-4038-a883-e0ec188212c0 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:26.979357 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 107/214] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Aug 24 17:07:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:26.986778 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c971a31d-cafb-4038-a883-e0ec188212c0 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:27.067673 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:27.086802 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4346a7-1925-46ea-8a34-c4adef5659f6 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:27.094727 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c971a31d-cafb-4038-a883-e0ec188212c0 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:27.119647 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:27.126321 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d10cad1-6d8b-4547-883e-b98e1e2aac4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:27.127510 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d10cad1-6d8b-4547-883e-b98e1e2aac4a None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:27.128180 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d10cad1-6d8b-4547-883e-b98e1e2aac4a None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:27.128763 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d10cad1-6d8b-4547-883e-b98e1e2aac4a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:27.129237 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f23dc26-cc7b-41b3-a9a2-4938b4b88dda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:27.129872 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f23dc26-cc7b-41b3-a9a2-4938b4b88dda None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:27.130154 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f23dc26-cc7b-41b3-a9a2-4938b4b88dda None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:27.130428 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f23dc26-cc7b-41b3-a9a2-4938b4b88dda None admin] PATH_INFO: `/v3/users/1419039d5f9e4550a09ae17ad5e771e5` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:27.131039 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d10cad1-6d8b-4547-883e-b98e1e2aac4a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:27.133049 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d10cad1-6d8b-4547-883e-b98e1e2aac4a None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:27.135622 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 108/215] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 24 17:07:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:27.141913 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d10cad1-6d8b-4547-883e-b98e1e2aac4a None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:27.145102 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f23dc26-cc7b-41b3-a9a2-4938b4b88dda None admin] RBAC: Authorizing `identity:get_user(user_id=1419039d5f9e4550a09ae17ad5e771e5)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:27.146713 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f23dc26-cc7b-41b3-a9a2-4938b4b88dda None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:27.148267 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 108/216] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 24 17:07:26 2022] GET /identity/v3/users/1419039d5f9e4550a09ae17ad5e771e5 => generated 297 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:07:27.153820 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f23dc26-cc7b-41b3-a9a2-4938b4b88dda None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:27.243712 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d10cad1-6d8b-4547-883e-b98e1e2aac4a None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:27.256901 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f23dc26-cc7b-41b3-a9a2-4938b4b88dda None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:27.279403 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3160201-a4d4-409d-85f1-6be94cc9bf0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:27.280383 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3160201-a4d4-409d-85f1-6be94cc9bf0f None admin] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:27.280863 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3160201-a4d4-409d-85f1-6be94cc9bf0f None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:27.281335 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3160201-a4d4-409d-85f1-6be94cc9bf0f None admin] PATH_INFO: `/v3/users` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:27.282180 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3160201-a4d4-409d-85f1-6be94cc9bf0f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:27.283831 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3160201-a4d4-409d-85f1-6be94cc9bf0f None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:27.292463 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74f0c46-1fca-4c4f-b447-fde1bd3bcd30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:27.293054 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74f0c46-1fca-4c4f-b447-fde1bd3bcd30 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:27.293336 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74f0c46-1fca-4c4f-b447-fde1bd3bcd30 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:27.293610 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74f0c46-1fca-4c4f-b447-fde1bd3bcd30 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:27.343444 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:27.365534 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74f0c46-1fca-4c4f-b447-fde1bd3bcd30 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:27.367715 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74f0c46-1fca-4c4f-b447-fde1bd3bcd30 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:27.419156 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 109/217] 10.222.0.18 () {60 vars in 1132 bytes} [Wed Aug 24 17:07:27 2022] POST /identity/v3/users => generated 297 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:07:27.421680 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d1b769-e20f-4969-b4f2-475e14b055cc None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:27.421970 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d1b769-e20f-4969-b4f2-475e14b055cc None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:27.422240 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d1b769-e20f-4969-b4f2-475e14b055cc None None] PATH_INFO: `/` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:27.428248 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 110/218] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:07:27 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:07:27.434159 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550c637e-9615-4a82-bb69-a9440603e612 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:27.434369 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550c637e-9615-4a82-bb69-a9440603e612 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:27.434587 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550c637e-9615-4a82-bb69-a9440603e612 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:27.477121 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-550c637e-9615-4a82-bb69-a9440603e612 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:07:27.479291 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-550c637e-9615-4a82-bb69-a9440603e612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:07:27.505673 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 109/219] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 24 17:07:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 17:07:27.512022 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74f0c46-1fca-4c4f-b447-fde1bd3bcd30 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:27.558291 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:27.563324 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:27.588032 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 111/220] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:07:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:07:27.594359 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be788c8f-819e-4cb0-8598-42db19f76560 None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:27.594886 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be788c8f-819e-4cb0-8598-42db19f76560 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:27.595375 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be788c8f-819e-4cb0-8598-42db19f76560 None None] PATH_INFO: `/` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:27.596995 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 112/221] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:07:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 17:07:27.602523 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a364bd-b641-4647-930d-5c156822ced6 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:27.606620 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a364bd-b641-4647-930d-5c156822ced6 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:27.606620 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a364bd-b641-4647-930d-5c156822ced6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:27.635079 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-e8a364bd-b641-4647-930d-5c156822ced6 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:07:27.641256 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-e8a364bd-b641-4647-930d-5c156822ced6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:07:27.648760 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74f0c46-1fca-4c4f-b447-fde1bd3bcd30 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:27.699783 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c951a15d-6b67-413f-80f2-9f016a938e27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:27.700428 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c951a15d-6b67-413f-80f2-9f016a938e27 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:27.700701 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c951a15d-6b67-413f-80f2-9f016a938e27 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:27.700969 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c951a15d-6b67-413f-80f2-9f016a938e27 None admin] PATH_INFO: `/v3/projects/94586413811643bb999468e84b659403` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:27.702869 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c951a15d-6b67-413f-80f2-9f016a938e27 None admin] RBAC: Authorizing `identity:get_project(project_id=94586413811643bb999468e84b659403)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:27.704343 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c951a15d-6b67-413f-80f2-9f016a938e27 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:27.705954 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 110/222] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 24 17:07:27 2022] GET /identity/v3/projects/94586413811643bb999468e84b659403 => generated 302 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:27.713284 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c951a15d-6b67-413f-80f2-9f016a938e27 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:27.734380 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 113/223] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:07:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 17:07:27.824818 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:27.828112 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c951a15d-6b67-413f-80f2-9f016a938e27 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:27.863261 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe4f51b-3d3f-4504-9cc9-83dad5ba85d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:27.863869 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe4f51b-3d3f-4504-9cc9-83dad5ba85d0 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:27.864101 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe4f51b-3d3f-4504-9cc9-83dad5ba85d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:27.864332 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe4f51b-3d3f-4504-9cc9-83dad5ba85d0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:27.865062 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe4f51b-3d3f-4504-9cc9-83dad5ba85d0 None admin] RBAC: Adding query filter params (role.id=60dc7e16aee746d5a3e8aa27d7d8525c, scope.project.id=94586413811643bb999468e84b659403, user.id=1419039d5f9e4550a09ae17ad5e771e5) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:07:27.865366 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe4f51b-3d3f-4504-9cc9-83dad5ba85d0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:27.866800 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe4f51b-3d3f-4504-9cc9-83dad5ba85d0 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:27.872784 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 111/224] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 24 17:07:27 2022] GET /identity/v3/role_assignments?user.id=1419039d5f9e4550a09ae17ad5e771e5&scope.project.id=94586413811643bb999468e84b659403&role.id=60dc7e16aee746d5a3e8aa27d7d8525c => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:07:27.877251 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a364bd-b641-4647-930d-5c156822ced6 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:27.990682 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a364bd-b641-4647-930d-5c156822ced6 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:28.009189 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:28.027590 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863bba0c-b838-4576-a395-28822171a389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:28.030669 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863bba0c-b838-4576-a395-28822171a389 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:28.030669 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863bba0c-b838-4576-a395-28822171a389 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:28.030669 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863bba0c-b838-4576-a395-28822171a389 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:28.031465 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe4f51b-3d3f-4504-9cc9-83dad5ba85d0 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:28.046756 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863bba0c-b838-4576-a395-28822171a389 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:07:28.054712 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-863bba0c-b838-4576-a395-28822171a389 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:07:28.054712 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:07:28.054712 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:07:28.054712 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:07:28.054712 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:07:28.054712 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:07:28.054712 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:07:28.054712 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:07:28.054712 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:07:28.054712 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:07:28.054712 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:07:28.054712 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:07:28.054712 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:07:28.054712 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:07:28.054712 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:07:28.054712 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:07:28.055782 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:07:28.055782 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:07:28.055782 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:07:28.055782 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:07:28.055782 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:07:28.055782 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:07:28.055782 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:07:28.055782 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:07:28.055782 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:07:28.055782 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:07:28.055782 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:07:28.055782 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application with Lock( Aug 24 17:07:28.055782 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:07:28.055782 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:07:28.055782 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:07:28.055782 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:07:28.056607 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:07:28.056607 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:07:28.056607 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:07:28.056607 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:07:28.056607 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:07:28.056607 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:07:28.056607 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:07:28.056607 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:07:28.056607 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:07:28.056607 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:07:28.056607 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:07:28.056607 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:07:28.056607 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 114/225] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Aug 24 17:07:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:07:28.064291 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863bba0c-b838-4576-a395-28822171a389 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:28.163439 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe4f51b-3d3f-4504-9cc9-83dad5ba85d0 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:28.182375 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863bba0c-b838-4576-a395-28822171a389 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:28.203050 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ba1b83-c429-4a18-aaa9-964295656dca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:28.204044 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ba1b83-c429-4a18-aaa9-964295656dca None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:28.204591 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ba1b83-c429-4a18-aaa9-964295656dca None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:28.205085 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ba1b83-c429-4a18-aaa9-964295656dca None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:28.217235 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ba1b83-c429-4a18-aaa9-964295656dca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:07:28.219023 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e9f37f-b2ed-4b3b-9803-b0c5d0c8656e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:28.219743 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e9f37f-b2ed-4b3b-9803-b0c5d0c8656e None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:28.220123 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e9f37f-b2ed-4b3b-9803-b0c5d0c8656e None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:28.220495 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e9f37f-b2ed-4b3b-9803-b0c5d0c8656e None admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:28.221268 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e9f37f-b2ed-4b3b-9803-b0c5d0c8656e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:07:28.221670 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e9f37f-b2ed-4b3b-9803-b0c5d0c8656e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:28.223184 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e9f37f-b2ed-4b3b-9803-b0c5d0c8656e None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:28.223928 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-39ba1b83-c429-4a18-aaa9-964295656dca None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 17:07:28.223928 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:07:28.223928 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:07:28.223928 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:07:28.223928 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:07:28.223928 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:07:28.223928 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:07:28.223928 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:07:28.223928 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:07:28.223928 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:07:28.223928 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:07:28.223928 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:07:28.223928 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:07:28.223928 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:07:28.223928 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:07:28.223928 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:07:28.224959 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:07:28.224959 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:07:28.224959 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:07:28.224959 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:07:28.224959 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:07:28.224959 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:07:28.224959 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:07:28.224959 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:07:28.224959 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:07:28.224959 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:07:28.224959 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:07:28.224959 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 24 17:07:28.224959 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:07:28.224959 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:07:28.224959 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:07:28.224959 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:07:28.225739 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:07:28.225739 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:07:28.225739 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:07:28.225739 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:07:28.225739 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:07:28.225739 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:07:28.225739 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:07:28.225739 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:07:28.225739 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:07:28.225739 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:07:28.225739 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 17:07:28.225739 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:07:28.233464 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 112/226] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Aug 24 17:07:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:07:28.234066 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 115/227] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 24 17:07:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:28.235343 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ba1b83-c429-4a18-aaa9-964295656dca None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:28.239573 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e9f37f-b2ed-4b3b-9803-b0c5d0c8656e None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:28.265172 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:28.283942 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:28.351250 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ba1b83-c429-4a18-aaa9-964295656dca None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:28.370782 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e9f37f-b2ed-4b3b-9803-b0c5d0c8656e None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:28.416818 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53744de6-cbe7-4aa1-af91-aea01788eb28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:28.417764 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53744de6-cbe7-4aa1-af91-aea01788eb28 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:28.420641 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53744de6-cbe7-4aa1-af91-aea01788eb28 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:28.422369 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53744de6-cbe7-4aa1-af91-aea01788eb28 None admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:28.423235 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53744de6-cbe7-4aa1-af91-aea01788eb28 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:07:28.424011 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ad0f2d-0456-4f62-be73-be03a7b0dc78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:28.424497 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53744de6-cbe7-4aa1-af91-aea01788eb28 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:28.425282 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ad0f2d-0456-4f62-be73-be03a7b0dc78 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:28.426843 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53744de6-cbe7-4aa1-af91-aea01788eb28 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:28.427289 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ad0f2d-0456-4f62-be73-be03a7b0dc78 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:28.427722 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ad0f2d-0456-4f62-be73-be03a7b0dc78 None admin] PATH_INFO: `/v3/users/815d5ea515c943b3ae9100dfc2acc10a` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:28.430500 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ad0f2d-0456-4f62-be73-be03a7b0dc78 None admin] RBAC: Authorizing `identity:get_user(user_id=815d5ea515c943b3ae9100dfc2acc10a)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:28.432124 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ad0f2d-0456-4f62-be73-be03a7b0dc78 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:28.434725 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 116/228] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 24 17:07:28 2022] GET /identity/v3/users/815d5ea515c943b3ae9100dfc2acc10a => generated 283 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:28.441417 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 113/229] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 24 17:07:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:28.446863 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ad0f2d-0456-4f62-be73-be03a7b0dc78 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:28.451638 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53744de6-cbe7-4aa1-af91-aea01788eb28 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:28.582001 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ad0f2d-0456-4f62-be73-be03a7b0dc78 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:28.590572 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:28.625206 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:28.636796 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0ac97b-76be-43e5-bd09-c3303f42a6a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:28.637442 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0ac97b-76be-43e5-bd09-c3303f42a6a9 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:28.637719 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0ac97b-76be-43e5-bd09-c3303f42a6a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:28.637992 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0ac97b-76be-43e5-bd09-c3303f42a6a9 None admin] PATH_INFO: `/v3/projects/94586413811643bb999468e84b659403` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:28.639879 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0ac97b-76be-43e5-bd09-c3303f42a6a9 None admin] RBAC: Authorizing `identity:get_project(project_id=94586413811643bb999468e84b659403)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:28.642613 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0ac97b-76be-43e5-bd09-c3303f42a6a9 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:28.645335 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 117/230] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 24 17:07:28 2022] GET /identity/v3/projects/94586413811643bb999468e84b659403 => generated 302 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:28.651038 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0ac97b-76be-43e5-bd09-c3303f42a6a9 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:28.652262 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53744de6-cbe7-4aa1-af91-aea01788eb28 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:28.728136 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bc537b-a422-4cfd-bb7f-336f11d3a2ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:28.728697 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bc537b-a422-4cfd-bb7f-336f11d3a2ba None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:28.728977 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bc537b-a422-4cfd-bb7f-336f11d3a2ba None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:28.729252 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bc537b-a422-4cfd-bb7f-336f11d3a2ba None admin] PATH_INFO: `/v3/users/815d5ea515c943b3ae9100dfc2acc10a` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:28.730413 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bc537b-a422-4cfd-bb7f-336f11d3a2ba None admin] RBAC: Authorizing `identity:get_user(user_id=815d5ea515c943b3ae9100dfc2acc10a)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:28.731929 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bc537b-a422-4cfd-bb7f-336f11d3a2ba None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:28.738234 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 114/231] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 24 17:07:28 2022] GET /identity/v3/users/815d5ea515c943b3ae9100dfc2acc10a => generated 283 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:28.743817 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bc537b-a422-4cfd-bb7f-336f11d3a2ba None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:28.853289 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0ac97b-76be-43e5-bd09-c3303f42a6a9 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:28.894818 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee39e3a-b0ce-4bd5-8c2d-f66b5e223547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:28.900406 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee39e3a-b0ce-4bd5-8c2d-f66b5e223547 None admin] REQUEST_METHOD: `PUT` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:28.900926 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee39e3a-b0ce-4bd5-8c2d-f66b5e223547 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:28.901420 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee39e3a-b0ce-4bd5-8c2d-f66b5e223547 None admin] PATH_INFO: `/v3/projects/94586413811643bb999468e84b659403/users/815d5ea515c943b3ae9100dfc2acc10a/roles/c5346bc54abf445f9900edf94264c850` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:28.904594 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee39e3a-b0ce-4bd5-8c2d-f66b5e223547 None admin] RBAC: Authorizing `identity:create_grant(project_id=94586413811643bb999468e84b659403, user_id=815d5ea515c943b3ae9100dfc2acc10a, role_id=c5346bc54abf445f9900edf94264c850)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:28.908371 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee39e3a-b0ce-4bd5-8c2d-f66b5e223547 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:28.933849 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bc537b-a422-4cfd-bb7f-336f11d3a2ba None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:28.970578 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec3f9b0-9dee-46d9-a95b-4100b989e77f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:28.971245 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec3f9b0-9dee-46d9-a95b-4100b989e77f None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:28.971524 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec3f9b0-9dee-46d9-a95b-4100b989e77f None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:28.971798 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec3f9b0-9dee-46d9-a95b-4100b989e77f None admin] PATH_INFO: `/v3/projects/94586413811643bb999468e84b659403` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:28.973028 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec3f9b0-9dee-46d9-a95b-4100b989e77f None admin] RBAC: Authorizing `identity:get_project(project_id=94586413811643bb999468e84b659403)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:28.974502 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec3f9b0-9dee-46d9-a95b-4100b989e77f None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:28.976378 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 115/232] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 24 17:07:28 2022] GET /identity/v3/projects/94586413811643bb999468e84b659403 => generated 302 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:28.992639 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 118/233] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Aug 24 17:07:28 2022] PUT /identity/v3/projects/94586413811643bb999468e84b659403/users/815d5ea515c943b3ae9100dfc2acc10a/roles/c5346bc54abf445f9900edf94264c850 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:07:28.994465 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec3f9b0-9dee-46d9-a95b-4100b989e77f None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:29.024484 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:29.127589 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec3f9b0-9dee-46d9-a95b-4100b989e77f None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:29.166136 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1567c6-75c9-4055-b925-2f0eba19c5d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:29.167305 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1567c6-75c9-4055-b925-2f0eba19c5d3 None admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:29.167797 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1567c6-75c9-4055-b925-2f0eba19c5d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:29.168346 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1567c6-75c9-4055-b925-2f0eba19c5d3 None admin] PATH_INFO: `/v3/projects/94586413811643bb999468e84b659403/users/815d5ea515c943b3ae9100dfc2acc10a/roles/af19a7a2fae5422da5bd7daea4bbfccf` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:29.171595 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1567c6-75c9-4055-b925-2f0eba19c5d3 None admin] RBAC: Authorizing `identity:create_grant(project_id=94586413811643bb999468e84b659403, user_id=815d5ea515c943b3ae9100dfc2acc10a, role_id=af19a7a2fae5422da5bd7daea4bbfccf)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:29.173374 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1567c6-75c9-4055-b925-2f0eba19c5d3 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:29.279762 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 116/234] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Aug 24 17:07:28 2022] PUT /identity/v3/projects/94586413811643bb999468e84b659403/users/815d5ea515c943b3ae9100dfc2acc10a/roles/af19a7a2fae5422da5bd7daea4bbfccf => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:07:29.841391 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c525bf1e-a882-4e45-9a91-80c97dedb01f None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:29.841939 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c525bf1e-a882-4e45-9a91-80c97dedb01f None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:29.841939 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c525bf1e-a882-4e45-9a91-80c97dedb01f None None] PATH_INFO: `/` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:29.843630 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 119/235] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:07:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:07:29.849032 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0883de-a1d7-4c30-9e86-dbe65393e00e None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:29.849319 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0883de-a1d7-4c30-9e86-dbe65393e00e None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:29.849591 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0883de-a1d7-4c30-9e86-dbe65393e00e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:29.875798 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-5f0883de-a1d7-4c30-9e86-dbe65393e00e None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:07:29.876985 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-5f0883de-a1d7-4c30-9e86-dbe65393e00e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:07:29.960431 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 117/236] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:07:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 17:07:30.022214 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e501d4f-ee5e-4a5c-a822-dcef8f54bd60 None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:30.022532 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e501d4f-ee5e-4a5c-a822-dcef8f54bd60 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:30.022764 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e501d4f-ee5e-4a5c-a822-dcef8f54bd60 None None] PATH_INFO: `/` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:30.024423 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 120/237] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:07:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:07:30.029461 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839a263d-a117-4cc2-b61a-85d7a627816c None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:30.029744 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839a263d-a117-4cc2-b61a-85d7a627816c None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:30.030030 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839a263d-a117-4cc2-b61a-85d7a627816c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:30.049756 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-839a263d-a117-4cc2-b61a-85d7a627816c None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:07:30.050941 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-839a263d-a117-4cc2-b61a-85d7a627816c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:07:30.064521 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:30.145279 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 118/238] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:07:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:07:30.284366 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f05443-6481-41eb-8475-a36cbd488346 None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:30.284695 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f05443-6481-41eb-8475-a36cbd488346 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:30.284918 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f05443-6481-41eb-8475-a36cbd488346 None None] PATH_INFO: `/` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:30.286270 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 121/239] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:07:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:07:30.291553 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5bf1c3-6f72-4bde-a993-6ce4771e79e1 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:30.291845 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5bf1c3-6f72-4bde-a993-6ce4771e79e1 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:30.292120 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5bf1c3-6f72-4bde-a993-6ce4771e79e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:30.314867 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-ec5bf1c3-6f72-4bde-a993-6ce4771e79e1 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:07:30.316081 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-ec5bf1c3-6f72-4bde-a993-6ce4771e79e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:07:30.347997 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f05443-6481-41eb-8475-a36cbd488346 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:30.387247 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:30.424064 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 119/240] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:07:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 17:07:30.484553 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f05443-6481-41eb-8475-a36cbd488346 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:30.534627 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e5c4c3-6d7c-46be-af10-accb3468a432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:30.535307 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e5c4c3-6d7c-46be-af10-accb3468a432 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:30.535585 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e5c4c3-6d7c-46be-af10-accb3468a432 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:30.535855 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e5c4c3-6d7c-46be-af10-accb3468a432 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:30.552832 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e5c4c3-6d7c-46be-af10-accb3468a432 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:07:30.560134 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-91e5c4c3-6d7c-46be-af10-accb3468a432 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 17:07:30.560134 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:07:30.560134 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:07:30.560134 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:07:30.560134 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:07:30.560134 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:07:30.560134 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:07:30.560134 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:07:30.560134 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:07:30.560134 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:07:30.560134 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:07:30.560134 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:07:30.560134 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:07:30.560134 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:07:30.560134 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:07:30.560134 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:07:30.561231 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:07:30.561231 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:07:30.561231 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:07:30.561231 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:07:30.561231 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:07:30.561231 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:07:30.561231 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:07:30.561231 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:07:30.561231 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:07:30.561231 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:07:30.561231 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:07:30.561231 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application with Lock( Aug 24 17:07:30.561231 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:07:30.561231 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:07:30.561231 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:07:30.561231 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:07:30.562218 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:07:30.562218 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:07:30.562218 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:07:30.562218 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:07:30.562218 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:07:30.562218 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:07:30.562218 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:07:30.562218 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:07:30.562218 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:07:30.562218 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:07:30.562218 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 17:07:30.562218 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:07:30.562218 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 122/241] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 24 17:07:30 2022] GET /identity/v3/roles/reader => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:07:30.565020 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5bf1c3-6f72-4bde-a993-6ce4771e79e1 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:30.585748 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e5c4c3-6d7c-46be-af10-accb3468a432 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:30.705073 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:30.719023 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:30.731360 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e5c4c3-6d7c-46be-af10-accb3468a432 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:30.751545 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5bf1c3-6f72-4bde-a993-6ce4771e79e1 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:30.778817 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc499e0-919e-43d1-92d2-bb07dd1bb7d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:30.780034 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc499e0-919e-43d1-92d2-bb07dd1bb7d0 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:30.780442 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc499e0-919e-43d1-92d2-bb07dd1bb7d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:30.780727 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc499e0-919e-43d1-92d2-bb07dd1bb7d0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:30.782022 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc499e0-919e-43d1-92d2-bb07dd1bb7d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:30.784157 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc499e0-919e-43d1-92d2-bb07dd1bb7d0 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:30.786935 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 123/242] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 24 17:07:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:07:30.787784 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc499e0-919e-43d1-92d2-bb07dd1bb7d0 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:30.790209 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904a1ece-39a1-4078-a65e-f0b8d2c8fc39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:30.790968 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904a1ece-39a1-4078-a65e-f0b8d2c8fc39 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:30.791257 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904a1ece-39a1-4078-a65e-f0b8d2c8fc39 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:30.791631 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904a1ece-39a1-4078-a65e-f0b8d2c8fc39 None admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:30.792306 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904a1ece-39a1-4078-a65e-f0b8d2c8fc39 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:07:30.792695 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904a1ece-39a1-4078-a65e-f0b8d2c8fc39 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:30.793996 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904a1ece-39a1-4078-a65e-f0b8d2c8fc39 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:30.804450 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 120/243] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Aug 24 17:07:30 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:30.805419 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904a1ece-39a1-4078-a65e-f0b8d2c8fc39 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:30.914098 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904a1ece-39a1-4078-a65e-f0b8d2c8fc39 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:30.914286 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc499e0-919e-43d1-92d2-bb07dd1bb7d0 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:30.950667 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a4a461-f5cf-4342-98db-14696883db98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:30.951448 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a4a461-f5cf-4342-98db-14696883db98 None admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:30.951731 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a4a461-f5cf-4342-98db-14696883db98 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:30.952103 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a4a461-f5cf-4342-98db-14696883db98 None admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:30.952751 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a4a461-f5cf-4342-98db-14696883db98 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:30.954195 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a4a461-f5cf-4342-98db-14696883db98 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:30.956975 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af88066-2a69-4671-914d-6cae7e544462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:30.957560 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af88066-2a69-4671-914d-6cae7e544462 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:30.957844 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af88066-2a69-4671-914d-6cae7e544462 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:30.958120 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af88066-2a69-4671-914d-6cae7e544462 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:30.962247 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:30.970442 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af88066-2a69-4671-914d-6cae7e544462 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:07:30.975676 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-1af88066-2a69-4671-914d-6cae7e544462 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:07:30.975676 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:07:30.975676 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:07:30.975676 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:07:30.975676 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:07:30.975676 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:07:30.975676 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:07:30.975676 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:07:30.975676 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:07:30.975676 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:07:30.975676 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:07:30.975676 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:07:30.975676 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:07:30.975676 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:07:30.975676 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:07:30.975676 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:07:30.976539 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:07:30.976539 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:07:30.976539 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:07:30.976539 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:07:30.976539 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:07:30.976539 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:07:30.976539 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:07:30.976539 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:07:30.976539 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:07:30.976539 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:07:30.976539 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:07:30.976539 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application with Lock( Aug 24 17:07:30.976539 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:07:30.976539 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:07:30.976539 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:07:30.976539 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:07:30.977341 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:07:30.977341 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:07:30.977341 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:07:30.977341 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:07:30.977341 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:07:30.977341 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:07:30.977341 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:07:30.977341 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:07:30.977341 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:07:30.977341 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:07:30.977341 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:07:30.977341 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:07:30.977341 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 124/244] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 24 17:07:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:07:30.978000 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af88066-2a69-4671-914d-6cae7e544462 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:31.006118 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:31.047778 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 121/245] 10.222.0.18 () {60 vars in 1132 bytes} [Wed Aug 24 17:07:30 2022] POST /identity/v3/users => generated 293 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:07:31.048944 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a4a461-f5cf-4342-98db-14696883db98 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:31.109709 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af88066-2a69-4671-914d-6cae7e544462 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:31.158026 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcaf41b0-4678-4e95-9372-7b076624f04c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:31.159103 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaf41b0-4678-4e95-9372-7b076624f04c None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:31.159614 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaf41b0-4678-4e95-9372-7b076624f04c None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:31.160084 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaf41b0-4678-4e95-9372-7b076624f04c None admin] PATH_INFO: `/v3/users/a4abe9b8215c417a804ce7286737ac6f` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:31.175209 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcaf41b0-4678-4e95-9372-7b076624f04c None admin] RBAC: Authorizing `identity:get_user(user_id=a4abe9b8215c417a804ce7286737ac6f)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:31.176574 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a4a461-f5cf-4342-98db-14696883db98 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:31.177297 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcaf41b0-4678-4e95-9372-7b076624f04c None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:31.179291 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 125/246] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 24 17:07:30 2022] GET /identity/v3/users/a4abe9b8215c417a804ce7286737ac6f => generated 297 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:31.184889 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcaf41b0-4678-4e95-9372-7b076624f04c None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:31.225082 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9f20b0-4d9c-4715-a57c-bb5d3f2592f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:31.225635 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9f20b0-4d9c-4715-a57c-bb5d3f2592f7 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:31.225922 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9f20b0-4d9c-4715-a57c-bb5d3f2592f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:31.226198 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9f20b0-4d9c-4715-a57c-bb5d3f2592f7 None admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:31.226891 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9f20b0-4d9c-4715-a57c-bb5d3f2592f7 None admin] RBAC: Adding query filter params (name=member) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:07:31.227220 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9f20b0-4d9c-4715-a57c-bb5d3f2592f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:31.228693 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9f20b0-4d9c-4715-a57c-bb5d3f2592f7 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:31.245249 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 122/247] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Aug 24 17:07:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:31.248596 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9f20b0-4d9c-4715-a57c-bb5d3f2592f7 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:31.305468 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:31.331259 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcaf41b0-4678-4e95-9372-7b076624f04c None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:31.343264 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:31.372193 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ed07f4-9f8d-441a-aa9d-ec82ee2b5c43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:31.373659 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ed07f4-9f8d-441a-aa9d-ec82ee2b5c43 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:31.374164 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ed07f4-9f8d-441a-aa9d-ec82ee2b5c43 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:31.374635 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ed07f4-9f8d-441a-aa9d-ec82ee2b5c43 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:31.407625 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9f20b0-4d9c-4715-a57c-bb5d3f2592f7 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:31.446868 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbb5c40-0dd4-477a-af48-80e0be184944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:31.447845 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbb5c40-0dd4-477a-af48-80e0be184944 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:31.448303 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbb5c40-0dd4-477a-af48-80e0be184944 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:31.448777 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbb5c40-0dd4-477a-af48-80e0be184944 None admin] PATH_INFO: `/v3/users/1419039d5f9e4550a09ae17ad5e771e5` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:31.450232 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbb5c40-0dd4-477a-af48-80e0be184944 None admin] RBAC: Authorizing `identity:get_user(user_id=1419039d5f9e4550a09ae17ad5e771e5)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:31.452537 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbb5c40-0dd4-477a-af48-80e0be184944 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:31.454351 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 123/248] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 24 17:07:31 2022] GET /identity/v3/users/1419039d5f9e4550a09ae17ad5e771e5 => generated 297 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:31.459952 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbb5c40-0dd4-477a-af48-80e0be184944 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:31.462951 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ed07f4-9f8d-441a-aa9d-ec82ee2b5c43 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:31.465214 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ed07f4-9f8d-441a-aa9d-ec82ee2b5c43 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:31.570420 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:31.589863 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbb5c40-0dd4-477a-af48-80e0be184944 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:31.591271 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 126/249] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 24 17:07:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 17:07:31.592694 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c6f04c-a8d2-497f-8341-0386f0abeef9 None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:31.593635 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c6f04c-a8d2-497f-8341-0386f0abeef9 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:31.593635 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c6f04c-a8d2-497f-8341-0386f0abeef9 None None] PATH_INFO: `/` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:31.594794 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 127/250] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:07:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:07:31.600034 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c6f04c-a8d2-497f-8341-0386f0abeef9 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:31.614400 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:31.632574 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c23b01-179b-47ce-908e-c742561244aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:31.633263 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c23b01-179b-47ce-908e-c742561244aa None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:31.633508 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c23b01-179b-47ce-908e-c742561244aa None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:31.633783 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c23b01-179b-47ce-908e-c742561244aa None admin] PATH_INFO: `/v3/projects/94586413811643bb999468e84b659403` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:31.635639 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c23b01-179b-47ce-908e-c742561244aa None admin] RBAC: Authorizing `identity:get_project(project_id=94586413811643bb999468e84b659403)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:31.637107 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c23b01-179b-47ce-908e-c742561244aa None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:31.641556 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 124/251] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 24 17:07:31 2022] GET /identity/v3/projects/94586413811643bb999468e84b659403 => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:07:31.643188 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5112310e-4da6-49ae-8cdb-18a3d34b2626 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:31.643575 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5112310e-4da6-49ae-8cdb-18a3d34b2626 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:31.643957 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5112310e-4da6-49ae-8cdb-18a3d34b2626 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:31.667071 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-5112310e-4da6-49ae-8cdb-18a3d34b2626 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:07:31.669270 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-5112310e-4da6-49ae-8cdb-18a3d34b2626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:07:31.734916 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c6f04c-a8d2-497f-8341-0386f0abeef9 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:31.761949 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 125/252] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:07:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:07:31.762986 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5112310e-4da6-49ae-8cdb-18a3d34b2626 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:31.781164 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d861b0e6-0ee9-4401-bbe0-c1146c45f36b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:31.782309 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d861b0e6-0ee9-4401-bbe0-c1146c45f36b None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:31.782662 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d861b0e6-0ee9-4401-bbe0-c1146c45f36b None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:31.783041 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d861b0e6-0ee9-4401-bbe0-c1146c45f36b None admin] PATH_INFO: `/v3/projects/94586413811643bb999468e84b659403` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:31.784249 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d861b0e6-0ee9-4401-bbe0-c1146c45f36b None admin] RBAC: Authorizing `identity:get_project(project_id=94586413811643bb999468e84b659403)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:31.786317 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d861b0e6-0ee9-4401-bbe0-c1146c45f36b None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:31.788103 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 128/253] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 24 17:07:31 2022] GET /identity/v3/projects/94586413811643bb999468e84b659403 => generated 302 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:31.794584 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d861b0e6-0ee9-4401-bbe0-c1146c45f36b None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:31.858202 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:31.920505 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5112310e-4da6-49ae-8cdb-18a3d34b2626 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:31.948382 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d861b0e6-0ee9-4401-bbe0-c1146c45f36b None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:31.952551 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:31.980751 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e08785-1f37-43fc-9a6d-e574cb890699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:31.981441 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e08785-1f37-43fc-9a6d-e574cb890699 None admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:31.981708 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e08785-1f37-43fc-9a6d-e574cb890699 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:31.982004 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e08785-1f37-43fc-9a6d-e574cb890699 None admin] PATH_INFO: `/v3/projects/94586413811643bb999468e84b659403/users/1419039d5f9e4550a09ae17ad5e771e5/roles/60dc7e16aee746d5a3e8aa27d7d8525c` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:31.988407 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e08785-1f37-43fc-9a6d-e574cb890699 None admin] RBAC: Authorizing `identity:create_grant(project_id=94586413811643bb999468e84b659403, user_id=1419039d5f9e4550a09ae17ad5e771e5, role_id=60dc7e16aee746d5a3e8aa27d7d8525c)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:31.989866 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e08785-1f37-43fc-9a6d-e574cb890699 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:32.006649 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72401649-79e8-437e-a1df-0c7806f278b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:32.010462 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72401649-79e8-437e-a1df-0c7806f278b3 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:32.010462 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72401649-79e8-437e-a1df-0c7806f278b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:32.010462 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72401649-79e8-437e-a1df-0c7806f278b3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:32.010462 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72401649-79e8-437e-a1df-0c7806f278b3 None admin] RBAC: Adding query filter params (role.id=7f65138ab599491194e7f91aa424ceee, scope.project.id=94586413811643bb999468e84b659403, user.id=a4abe9b8215c417a804ce7286737ac6f) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:07:32.010462 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72401649-79e8-437e-a1df-0c7806f278b3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:32.010462 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72401649-79e8-437e-a1df-0c7806f278b3 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:32.016232 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 129/254] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 24 17:07:31 2022] GET /identity/v3/role_assignments?user.id=a4abe9b8215c417a804ce7286737ac6f&scope.project.id=94586413811643bb999468e84b659403&role.id=7f65138ab599491194e7f91aa424ceee => generated 259 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:07:32.017490 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf986f80-d2a4-4075-b351-62b9c4bbfd6f None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:32.017780 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf986f80-d2a4-4075-b351-62b9c4bbfd6f None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:32.018057 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf986f80-d2a4-4075-b351-62b9c4bbfd6f None None] PATH_INFO: `/` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:32.019747 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 130/255] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:07:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:07:32.028564 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8174851b-2efb-49a0-87fd-1f36573d78ae None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:32.029599 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8174851b-2efb-49a0-87fd-1f36573d78ae None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:32.030042 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8174851b-2efb-49a0-87fd-1f36573d78ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:32.067330 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 126/256] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Aug 24 17:07:31 2022] PUT /identity/v3/projects/94586413811643bb999468e84b659403/users/1419039d5f9e4550a09ae17ad5e771e5/roles/60dc7e16aee746d5a3e8aa27d7d8525c => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:07:32.079720 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-8174851b-2efb-49a0-87fd-1f36573d78ae None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:07:32.081515 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-8174851b-2efb-49a0-87fd-1f36573d78ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:07:32.180650 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 131/257] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:07:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:07:32.234437 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e08785-1f37-43fc-9a6d-e574cb890699 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:32.365738 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e08785-1f37-43fc-9a6d-e574cb890699 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:32.418328 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acba0713-7968-4165-bd9b-412df6b17195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:32.419373 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acba0713-7968-4165-bd9b-412df6b17195 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:32.420151 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acba0713-7968-4165-bd9b-412df6b17195 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:32.420653 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acba0713-7968-4165-bd9b-412df6b17195 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:32.435337 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acba0713-7968-4165-bd9b-412df6b17195 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:07:32.440408 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-acba0713-7968-4165-bd9b-412df6b17195 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:07:32.440408 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:07:32.440408 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:07:32.440408 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:07:32.440408 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:07:32.440408 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:07:32.440408 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:07:32.440408 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:07:32.440408 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:07:32.440408 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:07:32.440408 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:07:32.440408 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:07:32.440408 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:07:32.440408 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:07:32.440408 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:07:32.440408 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:07:32.441301 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:07:32.441301 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:07:32.441301 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:07:32.441301 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:07:32.441301 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:07:32.441301 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:07:32.441301 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:07:32.441301 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:07:32.441301 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:07:32.441301 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:07:32.441301 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:07:32.441301 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 24 17:07:32.441301 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:07:32.441301 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:07:32.441301 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:07:32.441301 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:07:32.442091 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:07:32.442091 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:07:32.442091 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:07:32.442091 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:07:32.442091 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:07:32.442091 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:07:32.442091 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:07:32.442091 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:07:32.442091 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:07:32.442091 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:07:32.442091 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:07:32.442091 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:07:32.444526 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 127/258] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Aug 24 17:07:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:07:32.450545 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8174851b-2efb-49a0-87fd-1f36573d78ae None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:32.496103 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:32.560058 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acba0713-7968-4165-bd9b-412df6b17195 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:32.583463 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8174851b-2efb-49a0-87fd-1f36573d78ae None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:32.629676 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:32.650292 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeed2515-646a-41f0-b104-2bbdc952f1aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:32.650929 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeed2515-646a-41f0-b104-2bbdc952f1aa None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:32.651219 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeed2515-646a-41f0-b104-2bbdc952f1aa None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:32.651488 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeed2515-646a-41f0-b104-2bbdc952f1aa None admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:32.652163 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeed2515-646a-41f0-b104-2bbdc952f1aa None admin] RBAC: Adding query filter params (name=admin) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:07:32.652463 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeed2515-646a-41f0-b104-2bbdc952f1aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:32.653842 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeed2515-646a-41f0-b104-2bbdc952f1aa None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:32.668885 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 132/259] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 24 17:07:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:32.676512 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeed2515-646a-41f0-b104-2bbdc952f1aa None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:32.741239 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acba0713-7968-4165-bd9b-412df6b17195 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:32.795001 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9d6a10-c149-41d6-8ecc-ddadf3c9974a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:32.795934 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9d6a10-c149-41d6-8ecc-ddadf3c9974a None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:32.796219 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9d6a10-c149-41d6-8ecc-ddadf3c9974a None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:32.796494 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9d6a10-c149-41d6-8ecc-ddadf3c9974a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:32.808636 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9d6a10-c149-41d6-8ecc-ddadf3c9974a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:07:32.814946 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-ab9d6a10-c149-41d6-8ecc-ddadf3c9974a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 17:07:32.814946 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:07:32.814946 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:07:32.814946 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:07:32.814946 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:07:32.814946 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:07:32.814946 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:07:32.814946 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:07:32.814946 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:07:32.814946 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:07:32.814946 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:07:32.814946 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:07:32.814946 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:07:32.814946 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:07:32.814946 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:07:32.814946 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:07:32.815904 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:07:32.815904 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:07:32.815904 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:07:32.815904 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:07:32.815904 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:07:32.815904 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:07:32.815904 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:07:32.815904 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:07:32.815904 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:07:32.815904 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:07:32.815904 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:07:32.815904 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 24 17:07:32.815904 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:07:32.815904 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:07:32.815904 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:07:32.815904 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:07:32.816706 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:07:32.816706 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:07:32.816706 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:07:32.816706 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:07:32.816706 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:07:32.816706 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:07:32.816706 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:07:32.816706 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:07:32.816706 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:07:32.816706 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:07:32.816706 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 17:07:32.816706 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:07:32.816706 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 128/260] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Aug 24 17:07:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:07:32.828816 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9d6a10-c149-41d6-8ecc-ddadf3c9974a None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:32.875860 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeed2515-646a-41f0-b104-2bbdc952f1aa None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:32.878328 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:32.916917 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0a86a7-2954-4c13-af5d-5d7e69846392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:32.917563 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0a86a7-2954-4c13-af5d-5d7e69846392 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:32.917844 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0a86a7-2954-4c13-af5d-5d7e69846392 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:32.918122 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0a86a7-2954-4c13-af5d-5d7e69846392 None admin] PATH_INFO: `/v3/users/815d5ea515c943b3ae9100dfc2acc10a` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:32.919289 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0a86a7-2954-4c13-af5d-5d7e69846392 None admin] RBAC: Authorizing `identity:get_user(user_id=815d5ea515c943b3ae9100dfc2acc10a)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:32.920761 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0a86a7-2954-4c13-af5d-5d7e69846392 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:32.922412 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 133/261] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 24 17:07:32 2022] GET /identity/v3/users/815d5ea515c943b3ae9100dfc2acc10a => generated 283 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:32.928936 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0a86a7-2954-4c13-af5d-5d7e69846392 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:33.053201 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0a86a7-2954-4c13-af5d-5d7e69846392 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:33.064376 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:33.071778 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9d6a10-c149-41d6-8ecc-ddadf3c9974a None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:33.089475 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307c9f95-9285-4737-acc9-c876fa2542a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:33.090096 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307c9f95-9285-4737-acc9-c876fa2542a1 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:33.090378 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307c9f95-9285-4737-acc9-c876fa2542a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:33.090653 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307c9f95-9285-4737-acc9-c876fa2542a1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:33.134524 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:33.150866 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75b6485-0864-4c25-8a7b-97bd3eb2db39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:33.151685 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75b6485-0864-4c25-8a7b-97bd3eb2db39 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:33.151960 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75b6485-0864-4c25-8a7b-97bd3eb2db39 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:33.152229 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75b6485-0864-4c25-8a7b-97bd3eb2db39 None admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:33.152894 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75b6485-0864-4c25-8a7b-97bd3eb2db39 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:07:33.153192 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75b6485-0864-4c25-8a7b-97bd3eb2db39 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:33.154603 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75b6485-0864-4c25-8a7b-97bd3eb2db39 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:33.163179 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307c9f95-9285-4737-acc9-c876fa2542a1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:33.164718 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307c9f95-9285-4737-acc9-c876fa2542a1 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:33.173303 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 129/262] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 24 17:07:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:33.179062 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75b6485-0864-4c25-8a7b-97bd3eb2db39 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:33.276724 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 134/263] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 24 17:07:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 17:07:33.278117 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307c9f95-9285-4737-acc9-c876fa2542a1 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:33.349362 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:33.382731 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307c9f95-9285-4737-acc9-c876fa2542a1 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:33.413737 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75b6485-0864-4c25-8a7b-97bd3eb2db39 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:33.421021 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f528257b-2c44-4240-8595-0a15e0fae4a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:33.421667 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f528257b-2c44-4240-8595-0a15e0fae4a8 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:33.421946 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f528257b-2c44-4240-8595-0a15e0fae4a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:33.422220 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f528257b-2c44-4240-8595-0a15e0fae4a8 None admin] PATH_INFO: `/v3/projects/94586413811643bb999468e84b659403` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:33.424147 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f528257b-2c44-4240-8595-0a15e0fae4a8 None admin] RBAC: Authorizing `identity:get_project(project_id=94586413811643bb999468e84b659403)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:33.429484 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f528257b-2c44-4240-8595-0a15e0fae4a8 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:33.431483 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 135/264] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 24 17:07:33 2022] GET /identity/v3/projects/94586413811643bb999468e84b659403 => generated 302 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:07:33.439068 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f528257b-2c44-4240-8595-0a15e0fae4a8 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:33.501221 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258571cf-4201-43b2-95d8-55a107f3aeda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:33.501863 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258571cf-4201-43b2-95d8-55a107f3aeda None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:33.502141 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258571cf-4201-43b2-95d8-55a107f3aeda None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:33.502414 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258571cf-4201-43b2-95d8-55a107f3aeda None admin] PATH_INFO: `/v3/users/815d5ea515c943b3ae9100dfc2acc10a` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:33.507670 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258571cf-4201-43b2-95d8-55a107f3aeda None admin] RBAC: Authorizing `identity:get_user(user_id=815d5ea515c943b3ae9100dfc2acc10a)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:33.509137 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258571cf-4201-43b2-95d8-55a107f3aeda None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:33.510758 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 130/265] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 24 17:07:33 2022] GET /identity/v3/users/815d5ea515c943b3ae9100dfc2acc10a => generated 283 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:07:33.516806 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258571cf-4201-43b2-95d8-55a107f3aeda None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:33.565629 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f528257b-2c44-4240-8595-0a15e0fae4a8 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:33.609147 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d49350-bbbd-4e68-8402-7688f33842c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:33.609992 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:33.610308 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d49350-bbbd-4e68-8402-7688f33842c4 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:33.610794 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d49350-bbbd-4e68-8402-7688f33842c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:33.611293 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d49350-bbbd-4e68-8402-7688f33842c4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:33.612291 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d49350-bbbd-4e68-8402-7688f33842c4 None admin] RBAC: Adding query filter params (role.id=c5346bc54abf445f9900edf94264c850, scope.project.id=94586413811643bb999468e84b659403, user.id=815d5ea515c943b3ae9100dfc2acc10a) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:07:33.612797 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d49350-bbbd-4e68-8402-7688f33842c4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:33.614516 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d49350-bbbd-4e68-8402-7688f33842c4 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:33.622289 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 136/266] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 24 17:07:33 2022] GET /identity/v3/role_assignments?user.id=815d5ea515c943b3ae9100dfc2acc10a&scope.project.id=94586413811643bb999468e84b659403&role.id=c5346bc54abf445f9900edf94264c850 => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:07:33.746253 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258571cf-4201-43b2-95d8-55a107f3aeda None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:33.789709 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715d75dc-f624-45e0-b4dc-6f32e3ed6cc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:33.790704 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715d75dc-f624-45e0-b4dc-6f32e3ed6cc9 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:33.791186 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715d75dc-f624-45e0-b4dc-6f32e3ed6cc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:33.791633 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715d75dc-f624-45e0-b4dc-6f32e3ed6cc9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:33.873163 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715d75dc-f624-45e0-b4dc-6f32e3ed6cc9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:33.875043 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715d75dc-f624-45e0-b4dc-6f32e3ed6cc9 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:33.911645 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:34.010373 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 131/267] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 24 17:07:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 494 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 17:07:34.017381 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d49350-bbbd-4e68-8402-7688f33842c4 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:34.021321 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:34.127322 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaef510d-9605-47c0-8ea2-3ac546c83e72 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:34.127807 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaef510d-9605-47c0-8ea2-3ac546c83e72 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:34.127807 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaef510d-9605-47c0-8ea2-3ac546c83e72 None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:34.129124 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 132/268] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:07:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 17:07:34.140335 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9bddb8-73a3-4612-bf3d-4c19f98a9f32 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:34.140720 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9bddb8-73a3-4612-bf3d-4c19f98a9f32 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:34.140876 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9bddb8-73a3-4612-bf3d-4c19f98a9f32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:34.167025 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-9a9bddb8-73a3-4612-bf3d-4c19f98a9f32 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:07:34.168280 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-9a9bddb8-73a3-4612-bf3d-4c19f98a9f32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:07:34.174701 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d49350-bbbd-4e68-8402-7688f33842c4 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:34.240654 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29275f59-0e6a-47bb-a9fb-328643e24d04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:34.241506 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29275f59-0e6a-47bb-a9fb-328643e24d04 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:34.241784 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29275f59-0e6a-47bb-a9fb-328643e24d04 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:34.242055 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29275f59-0e6a-47bb-a9fb-328643e24d04 None admin] PATH_INFO: `/v3/projects/94586413811643bb999468e84b659403` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:34.245316 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29275f59-0e6a-47bb-a9fb-328643e24d04 None admin] RBAC: Authorizing `identity:get_project(project_id=94586413811643bb999468e84b659403)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:34.248825 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29275f59-0e6a-47bb-a9fb-328643e24d04 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:34.254725 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 137/269] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 24 17:07:34 2022] GET /identity/v3/projects/94586413811643bb999468e84b659403 => generated 302 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:34.263991 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29275f59-0e6a-47bb-a9fb-328643e24d04 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:34.298823 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 133/270] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:07:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:07:34.369385 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:34.409286 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29275f59-0e6a-47bb-a9fb-328643e24d04 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:34.450670 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fc10d9-a26e-4f98-9b86-d36672f130f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:34.451320 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fc10d9-a26e-4f98-9b86-d36672f130f0 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:34.451605 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fc10d9-a26e-4f98-9b86-d36672f130f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:34.451882 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fc10d9-a26e-4f98-9b86-d36672f130f0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:34.452608 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fc10d9-a26e-4f98-9b86-d36672f130f0 None admin] RBAC: Adding query filter params (role.id=af19a7a2fae5422da5bd7daea4bbfccf, scope.project.id=94586413811643bb999468e84b659403, user.id=815d5ea515c943b3ae9100dfc2acc10a) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:07:34.452917 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fc10d9-a26e-4f98-9b86-d36672f130f0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:34.454352 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fc10d9-a26e-4f98-9b86-d36672f130f0 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:34.463299 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 138/271] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 24 17:07:34 2022] GET /identity/v3/role_assignments?user.id=815d5ea515c943b3ae9100dfc2acc10a&scope.project.id=94586413811643bb999468e84b659403&role.id=af19a7a2fae5422da5bd7daea4bbfccf => generated 608 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:07:34.517896 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7a74bb-5c85-4c86-be95-37aaab72d275 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:34.518239 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7a74bb-5c85-4c86-be95-37aaab72d275 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:34.518454 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7a74bb-5c85-4c86-be95-37aaab72d275 None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:34.519970 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 134/272] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:07:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:07:34.525740 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c240d3-7bc0-47d6-8a98-f5d5dcd7c437 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:34.526062 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c240d3-7bc0-47d6-8a98-f5d5dcd7c437 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:34.526303 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c240d3-7bc0-47d6-8a98-f5d5dcd7c437 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:34.550288 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-21c240d3-7bc0-47d6-8a98-f5d5dcd7c437 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:07:34.550409 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-21c240d3-7bc0-47d6-8a98-f5d5dcd7c437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:07:34.642447 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 139/273] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:07:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:07:34.642645 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7a74bb-5c85-4c86-be95-37aaab72d275 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:34.734200 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:34.779536 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7a74bb-5c85-4c86-be95-37aaab72d275 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:34.816525 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf15c82-48da-430b-868e-7c3a6aab15cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:34.817159 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf15c82-48da-430b-868e-7c3a6aab15cd None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:34.817438 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf15c82-48da-430b-868e-7c3a6aab15cd None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:34.817709 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf15c82-48da-430b-868e-7c3a6aab15cd None admin] PATH_INFO: `/v3/roles/member` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:34.829339 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf15c82-48da-430b-868e-7c3a6aab15cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:07:34.834590 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-eaf15c82-48da-430b-868e-7c3a6aab15cd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:07:34.834590 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:07:34.834590 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:07:34.834590 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:07:34.834590 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:07:34.834590 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:07:34.834590 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:07:34.834590 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:07:34.834590 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:07:34.834590 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:07:34.834590 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:07:34.834590 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:07:34.834590 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:07:34.834590 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:07:34.834590 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:07:34.834590 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:07:34.835523 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:07:34.835523 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:07:34.835523 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:07:34.835523 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:07:34.835523 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:07:34.835523 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:07:34.835523 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:07:34.835523 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:07:34.835523 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:07:34.835523 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:07:34.835523 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:07:34.835523 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 24 17:07:34.835523 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:07:34.835523 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:07:34.835523 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:07:34.835523 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:07:34.836316 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:07:34.836316 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:07:34.836316 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:07:34.836316 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:07:34.836316 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:07:34.836316 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:07:34.836316 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:07:34.836316 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:07:34.836316 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:07:34.836316 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:07:34.836316 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:07:34.836316 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:07:34.836316 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 135/274] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 24 17:07:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:07:34.841842 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c240d3-7bc0-47d6-8a98-f5d5dcd7c437 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:34.916632 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:34.935864 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf15c82-48da-430b-868e-7c3a6aab15cd None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:34.957423 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c240d3-7bc0-47d6-8a98-f5d5dcd7c437 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:35.002844 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d9e884-ba8c-4ac2-a8f0-6ad834bf3aac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:35.003898 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d9e884-ba8c-4ac2-a8f0-6ad834bf3aac None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:35.004376 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d9e884-ba8c-4ac2-a8f0-6ad834bf3aac None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:35.004841 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d9e884-ba8c-4ac2-a8f0-6ad834bf3aac None admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:35.005703 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d9e884-ba8c-4ac2-a8f0-6ad834bf3aac None admin] RBAC: Adding query filter params (name=member) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:07:35.006199 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d9e884-ba8c-4ac2-a8f0-6ad834bf3aac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:35.007819 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d9e884-ba8c-4ac2-a8f0-6ad834bf3aac None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:35.017048 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 140/275] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Aug 24 17:07:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:35.023724 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d9e884-ba8c-4ac2-a8f0-6ad834bf3aac None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:35.072389 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf15c82-48da-430b-868e-7c3a6aab15cd None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:35.093346 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:35.112678 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2039d60-719c-4b44-9e9a-4ec09c106ffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:35.113314 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2039d60-719c-4b44-9e9a-4ec09c106ffc None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:35.113595 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2039d60-719c-4b44-9e9a-4ec09c106ffc None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:35.113869 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2039d60-719c-4b44-9e9a-4ec09c106ffc None admin] PATH_INFO: `/v3/domains/default` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:35.115053 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2039d60-719c-4b44-9e9a-4ec09c106ffc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:35.116632 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2039d60-719c-4b44-9e9a-4ec09c106ffc None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:35.118398 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 136/276] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 24 17:07:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:07:35.119513 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e2df24-ab46-4d1d-98ce-3330d814e41f None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:35.119956 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e2df24-ab46-4d1d-98ce-3330d814e41f None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:35.120436 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e2df24-ab46-4d1d-98ce-3330d814e41f None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:35.122099 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 137/277] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:07:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:07:35.123671 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e2df24-ab46-4d1d-98ce-3330d814e41f None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:35.145917 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d9e884-ba8c-4ac2-a8f0-6ad834bf3aac None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:35.175987 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:35.184187 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b05f55-d167-47ca-bc52-ca44719e7ecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:35.186476 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b05f55-d167-47ca-bc52-ca44719e7ecf None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:35.186476 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b05f55-d167-47ca-bc52-ca44719e7ecf None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:35.186476 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b05f55-d167-47ca-bc52-ca44719e7ecf None admin] PATH_INFO: `/v3/users/c3c3a3eb18c649d696423f05615344a4` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:35.198357 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b05f55-d167-47ca-bc52-ca44719e7ecf None admin] RBAC: Authorizing `identity:get_user(user_id=c3c3a3eb18c649d696423f05615344a4)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:35.199832 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b05f55-d167-47ca-bc52-ca44719e7ecf None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:35.201472 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 141/278] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 24 17:07:35 2022] GET /identity/v3/users/c3c3a3eb18c649d696423f05615344a4 => generated 293 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:07:35.202751 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cae5f2-69f5-4028-b886-1bd52aeb94a4 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:35.203063 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cae5f2-69f5-4028-b886-1bd52aeb94a4 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:35.203341 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cae5f2-69f5-4028-b886-1bd52aeb94a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:35.221196 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-a5cae5f2-69f5-4028-b886-1bd52aeb94a4 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:07:35.222379 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-a5cae5f2-69f5-4028-b886-1bd52aeb94a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:07:35.236778 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e2df24-ab46-4d1d-98ce-3330d814e41f None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:35.269423 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4163f997-43bd-46ab-a5e2-cc91acc3b948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:35.269959 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4163f997-43bd-46ab-a5e2-cc91acc3b948 None admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:35.270241 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4163f997-43bd-46ab-a5e2-cc91acc3b948 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:35.270516 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4163f997-43bd-46ab-a5e2-cc91acc3b948 None admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:35.271189 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4163f997-43bd-46ab-a5e2-cc91acc3b948 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:35.272765 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4163f997-43bd-46ab-a5e2-cc91acc3b948 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:35.288646 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 142/279] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:07:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:07:35.289954 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3fb0ea-31c1-445f-8d63-bb59c5821b6c None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:35.290246 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3fb0ea-31c1-445f-8d63-bb59c5821b6c None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:35.295277 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3fb0ea-31c1-445f-8d63-bb59c5821b6c None None] PATH_INFO: `/` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:35.295277 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 143/280] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:07:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:07:35.295277 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3fb0ea-31c1-445f-8d63-bb59c5821b6c None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:35.379781 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 138/281] 10.222.0.18 () {60 vars in 1132 bytes} [Wed Aug 24 17:07:35 2022] POST /identity/v3/users => generated 293 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:07:35.381120 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d25818a-4976-423e-801d-963e9714fe6e None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:35.381514 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d25818a-4976-423e-801d-963e9714fe6e None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:35.382232 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d25818a-4976-423e-801d-963e9714fe6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:35.400844 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3fb0ea-31c1-445f-8d63-bb59c5821b6c None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:35.409568 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-1d25818a-4976-423e-801d-963e9714fe6e None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:07:35.412154 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-1d25818a-4976-423e-801d-963e9714fe6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:07:35.422696 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:35.450326 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d855a7a7-ab60-4205-8323-0db9c7e0853d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:35.451830 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d855a7a7-ab60-4205-8323-0db9c7e0853d None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:35.452111 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d855a7a7-ab60-4205-8323-0db9c7e0853d None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:35.452384 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d855a7a7-ab60-4205-8323-0db9c7e0853d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:35.453657 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d855a7a7-ab60-4205-8323-0db9c7e0853d None admin] RBAC: Adding query filter params (role.id=60dc7e16aee746d5a3e8aa27d7d8525c, user.id=c3c3a3eb18c649d696423f05615344a4, scope.system=all) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:07:35.453973 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d855a7a7-ab60-4205-8323-0db9c7e0853d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:35.455817 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d855a7a7-ab60-4205-8323-0db9c7e0853d None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:35.464673 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 144/282] 10.222.0.18 () {56 vars in 1308 bytes} [Wed Aug 24 17:07:35 2022] GET /identity/v3/role_assignments?user.id=c3c3a3eb18c649d696423f05615344a4&scope.system=all&role.id=60dc7e16aee746d5a3e8aa27d7d8525c => generated 226 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:35.472448 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:35.517079 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 139/283] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:07:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:07:35.574115 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d855a7a7-ab60-4205-8323-0db9c7e0853d None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:35.687065 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d855a7a7-ab60-4205-8323-0db9c7e0853d None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:35.725429 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e7e364-0041-4b87-a30a-46b481ffa70b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:35.726063 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e7e364-0041-4b87-a30a-46b481ffa70b None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:35.726388 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e7e364-0041-4b87-a30a-46b481ffa70b None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:35.726601 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e7e364-0041-4b87-a30a-46b481ffa70b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:35.737814 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e7e364-0041-4b87-a30a-46b481ffa70b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:07:35.742722 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-39e7e364-0041-4b87-a30a-46b481ffa70b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 17:07:35.742722 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:07:35.742722 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:07:35.742722 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:07:35.742722 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:07:35.742722 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:07:35.742722 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:07:35.742722 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:07:35.742722 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:07:35.742722 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:07:35.742722 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:07:35.742722 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:07:35.742722 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:07:35.742722 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:07:35.742722 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:07:35.742722 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:07:35.743637 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:07:35.743637 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:07:35.743637 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:07:35.743637 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:07:35.743637 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:07:35.743637 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:07:35.743637 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:07:35.743637 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:07:35.743637 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:07:35.743637 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:07:35.743637 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:07:35.743637 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application with Lock( Aug 24 17:07:35.743637 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:07:35.743637 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:07:35.743637 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:07:35.743637 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:07:35.744418 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:07:35.744418 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:07:35.744418 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:07:35.744418 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:07:35.744418 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:07:35.744418 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:07:35.744418 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:07:35.744418 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:07:35.744418 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:07:35.744418 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:07:35.744418 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 17:07:35.744418 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:07:35.744418 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 145/284] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 24 17:07:35 2022] GET /identity/v3/roles/reader => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:07:35.751968 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d25818a-4976-423e-801d-963e9714fe6e None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:35.807540 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e7e364-0041-4b87-a30a-46b481ffa70b None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:35.867287 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:35.884328 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d25818a-4976-423e-801d-963e9714fe6e None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:35.923162 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78b157d-96bf-4beb-8f45-348fe651222c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:35.923783 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78b157d-96bf-4beb-8f45-348fe651222c None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:35.924064 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78b157d-96bf-4beb-8f45-348fe651222c None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:35.924335 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78b157d-96bf-4beb-8f45-348fe651222c None admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:35.925115 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78b157d-96bf-4beb-8f45-348fe651222c None admin] RBAC: Adding query filter params (name=reader) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:07:35.925521 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78b157d-96bf-4beb-8f45-348fe651222c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:35.926930 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78b157d-96bf-4beb-8f45-348fe651222c None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:35.931275 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e7e364-0041-4b87-a30a-46b481ffa70b None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:35.936575 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 140/285] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Aug 24 17:07:35 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:35.942137 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78b157d-96bf-4beb-8f45-348fe651222c None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:35.967463 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee273891-a372-4a7b-8e53-65ced1b708e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:35.968132 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee273891-a372-4a7b-8e53-65ced1b708e8 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:35.968508 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee273891-a372-4a7b-8e53-65ced1b708e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:35.968781 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee273891-a372-4a7b-8e53-65ced1b708e8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:35.978535 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:35.980980 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee273891-a372-4a7b-8e53-65ced1b708e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:07:35.986554 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-ee273891-a372-4a7b-8e53-65ced1b708e8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:07:35.986554 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:07:35.986554 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:07:35.986554 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:07:35.986554 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:07:35.986554 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:07:35.986554 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:07:35.986554 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:07:35.986554 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:07:35.986554 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:07:35.986554 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:07:35.986554 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:07:35.986554 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:07:35.986554 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:07:35.986554 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:07:35.986554 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:07:35.992378 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:07:35.992378 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:07:35.992378 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:07:35.992378 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:07:35.992378 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:07:35.992378 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:07:35.992378 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:07:35.992378 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:07:35.992378 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:07:35.992378 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:07:35.992378 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:07:35.992378 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application with Lock( Aug 24 17:07:35.992378 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:07:35.992378 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:07:35.992378 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:07:35.992378 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:07:35.993178 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:07:35.993178 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:07:35.993178 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:07:35.993178 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:07:35.993178 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:07:35.993178 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:07:35.993178 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:07:35.993178 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:07:35.993178 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:07:35.993178 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:07:35.993178 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:07:35.993178 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:07:35.993178 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 146/286] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 24 17:07:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:07:36.000255 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee273891-a372-4a7b-8e53-65ced1b708e8 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:36.053657 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78b157d-96bf-4beb-8f45-348fe651222c None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:36.088177 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96965f81-5911-4194-8314-95e164044e74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:36.088919 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96965f81-5911-4194-8314-95e164044e74 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:36.089269 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96965f81-5911-4194-8314-95e164044e74 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:36.089616 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96965f81-5911-4194-8314-95e164044e74 None admin] PATH_INFO: `/v3/users/a4abe9b8215c417a804ce7286737ac6f` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:36.090830 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96965f81-5911-4194-8314-95e164044e74 None admin] RBAC: Authorizing `identity:get_user(user_id=a4abe9b8215c417a804ce7286737ac6f)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:36.092926 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96965f81-5911-4194-8314-95e164044e74 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:36.099037 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 141/287] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 24 17:07:35 2022] GET /identity/v3/users/a4abe9b8215c417a804ce7286737ac6f => generated 297 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:36.100497 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96965f81-5911-4194-8314-95e164044e74 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:36.167701 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee273891-a372-4a7b-8e53-65ced1b708e8 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:36.189065 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:36.211600 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2214dc20-8952-4a8a-8f38-be504a774d4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:36.212316 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2214dc20-8952-4a8a-8f38-be504a774d4b None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:36.212606 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2214dc20-8952-4a8a-8f38-be504a774d4b None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:36.212935 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2214dc20-8952-4a8a-8f38-be504a774d4b None admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:36.213611 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2214dc20-8952-4a8a-8f38-be504a774d4b None admin] RBAC: Adding query filter params (name=member) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:07:36.213913 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2214dc20-8952-4a8a-8f38-be504a774d4b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:36.215351 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2214dc20-8952-4a8a-8f38-be504a774d4b None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:36.225069 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 147/288] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Aug 24 17:07:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:36.230848 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2214dc20-8952-4a8a-8f38-be504a774d4b None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:36.349211 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2214dc20-8952-4a8a-8f38-be504a774d4b None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:36.385939 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4df753-a3ba-4b13-9542-ef9e1e31027d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:36.386573 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4df753-a3ba-4b13-9542-ef9e1e31027d None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:36.386854 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4df753-a3ba-4b13-9542-ef9e1e31027d None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:36.387162 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4df753-a3ba-4b13-9542-ef9e1e31027d None admin] PATH_INFO: `/v3/users/1419039d5f9e4550a09ae17ad5e771e5` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:36.388919 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4df753-a3ba-4b13-9542-ef9e1e31027d None admin] RBAC: Authorizing `identity:get_user(user_id=1419039d5f9e4550a09ae17ad5e771e5)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:36.389375 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96965f81-5911-4194-8314-95e164044e74 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:36.390373 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4df753-a3ba-4b13-9542-ef9e1e31027d None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:36.391069 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:36.393305 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 148/289] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 24 17:07:36 2022] GET /identity/v3/users/1419039d5f9e4550a09ae17ad5e771e5 => generated 297 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:36.400511 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4df753-a3ba-4b13-9542-ef9e1e31027d None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:36.435032 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3764488-054c-401a-b7c0-0281fe95d7dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:36.435751 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3764488-054c-401a-b7c0-0281fe95d7dc None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:36.436036 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3764488-054c-401a-b7c0-0281fe95d7dc None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:36.436314 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3764488-054c-401a-b7c0-0281fe95d7dc None admin] PATH_INFO: `/v3/projects/94586413811643bb999468e84b659403` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:36.442797 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3764488-054c-401a-b7c0-0281fe95d7dc None admin] RBAC: Authorizing `identity:get_project(project_id=94586413811643bb999468e84b659403)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:36.444303 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3764488-054c-401a-b7c0-0281fe95d7dc None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:36.450730 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 142/290] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 24 17:07:36 2022] GET /identity/v3/projects/94586413811643bb999468e84b659403 => generated 302 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:07:36.453024 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3764488-054c-401a-b7c0-0281fe95d7dc None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:36.459238 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:36.585081 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4df753-a3ba-4b13-9542-ef9e1e31027d None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:36.594287 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3764488-054c-401a-b7c0-0281fe95d7dc None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:36.637177 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0553a907-5a2c-4969-a47d-0acaf2e3997f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:36.637763 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0553a907-5a2c-4969-a47d-0acaf2e3997f None admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:36.638041 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0553a907-5a2c-4969-a47d-0acaf2e3997f None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:36.638340 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0553a907-5a2c-4969-a47d-0acaf2e3997f None admin] PATH_INFO: `/v3/projects/94586413811643bb999468e84b659403/users/a4abe9b8215c417a804ce7286737ac6f/roles/7f65138ab599491194e7f91aa424ceee` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:36.642043 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0553a907-5a2c-4969-a47d-0acaf2e3997f None admin] RBAC: Authorizing `identity:create_grant(project_id=94586413811643bb999468e84b659403, user_id=a4abe9b8215c417a804ce7286737ac6f, role_id=7f65138ab599491194e7f91aa424ceee)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:36.644311 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0553a907-5a2c-4969-a47d-0acaf2e3997f None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:36.659043 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9c42da-7bef-4c87-ae2a-9a5486d7b238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:36.659721 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9c42da-7bef-4c87-ae2a-9a5486d7b238 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:36.660001 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9c42da-7bef-4c87-ae2a-9a5486d7b238 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:36.664951 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9c42da-7bef-4c87-ae2a-9a5486d7b238 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:36.731371 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 143/291] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Aug 24 17:07:36 2022] PUT /identity/v3/projects/94586413811643bb999468e84b659403/users/a4abe9b8215c417a804ce7286737ac6f/roles/7f65138ab599491194e7f91aa424ceee => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:07:36.748856 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9c42da-7bef-4c87-ae2a-9a5486d7b238 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:36.750576 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9c42da-7bef-4c87-ae2a-9a5486d7b238 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:36.784786 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:36.893497 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 149/292] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 24 17:07:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 493 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 17:07:36.899084 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0553a907-5a2c-4969-a47d-0acaf2e3997f None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:36.933759 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:37.011077 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0553a907-5a2c-4969-a47d-0acaf2e3997f None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:37.052565 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714b79b9-5adb-4538-9ed2-1810d513517a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:37.053208 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714b79b9-5adb-4538-9ed2-1810d513517a None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:37.053484 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714b79b9-5adb-4538-9ed2-1810d513517a None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:37.053754 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714b79b9-5adb-4538-9ed2-1810d513517a None admin] PATH_INFO: `/v3/projects/94586413811643bb999468e84b659403` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:37.054878 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714b79b9-5adb-4538-9ed2-1810d513517a None admin] RBAC: Authorizing `identity:get_project(project_id=94586413811643bb999468e84b659403)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:37.056362 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714b79b9-5adb-4538-9ed2-1810d513517a None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:37.058161 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 144/293] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 24 17:07:36 2022] GET /identity/v3/projects/94586413811643bb999468e84b659403 => generated 302 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:37.063869 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9c42da-7bef-4c87-ae2a-9a5486d7b238 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:37.208032 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9c42da-7bef-4c87-ae2a-9a5486d7b238 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:37.251122 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:37.257748 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc83a85-2806-4543-9696-4d9aad32fa95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:37.258370 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc83a85-2806-4543-9696-4d9aad32fa95 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:37.258647 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc83a85-2806-4543-9696-4d9aad32fa95 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:37.258934 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc83a85-2806-4543-9696-4d9aad32fa95 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:37.259688 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc83a85-2806-4543-9696-4d9aad32fa95 None admin] RBAC: Adding query filter params (role.id=60dc7e16aee746d5a3e8aa27d7d8525c, scope.project.id=94586413811643bb999468e84b659403, user.id=1419039d5f9e4550a09ae17ad5e771e5) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:07:37.259996 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc83a85-2806-4543-9696-4d9aad32fa95 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:37.261436 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc83a85-2806-4543-9696-4d9aad32fa95 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:37.270128 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 150/294] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 24 17:07:37 2022] GET /identity/v3/role_assignments?user.id=1419039d5f9e4550a09ae17ad5e771e5&scope.project.id=94586413811643bb999468e84b659403&role.id=60dc7e16aee746d5a3e8aa27d7d8525c => generated 608 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:37.749853 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186bec0b-1b8c-43de-9ba2-4d0dc8e59f85 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:37.750753 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186bec0b-1b8c-43de-9ba2-4d0dc8e59f85 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:37.750753 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186bec0b-1b8c-43de-9ba2-4d0dc8e59f85 None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:37.752395 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 145/295] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:07:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:07:37.757827 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170b643f-c9e5-4643-a4f5-f43ae85e5c17 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:37.758115 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170b643f-c9e5-4643-a4f5-f43ae85e5c17 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:37.758388 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170b643f-c9e5-4643-a4f5-f43ae85e5c17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:37.771192 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a90d3ba-8bce-4a89-9cee-9d62298df52e None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:37.771523 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a90d3ba-8bce-4a89-9cee-9d62298df52e None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:37.771741 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a90d3ba-8bce-4a89-9cee-9d62298df52e None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:37.773965 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 146/296] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:07:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:07:37.778184 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-170b643f-c9e5-4643-a4f5-f43ae85e5c17 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:07:37.779699 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-170b643f-c9e5-4643-a4f5-f43ae85e5c17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:07:37.785330 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bf25a7-04f2-4384-bb7d-b69a17e059d9 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:37.785874 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bf25a7-04f2-4384-bb7d-b69a17e059d9 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:37.786349 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bf25a7-04f2-4384-bb7d-b69a17e059d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:37.809376 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-b9bf25a7-04f2-4384-bb7d-b69a17e059d9 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:07:37.811627 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-b9bf25a7-04f2-4384-bb7d-b69a17e059d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:07:37.882152 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 151/297] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:07:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:07:37.906868 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 147/298] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:07:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 17:07:38.117378 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7f0c97-e5ba-4fc7-aa28-1dd2daf998b7 None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:38.117659 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7f0c97-e5ba-4fc7-aa28-1dd2daf998b7 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:38.118239 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7f0c97-e5ba-4fc7-aa28-1dd2daf998b7 None None] PATH_INFO: `/` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:38.120134 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 152/299] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:07:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:07:38.126441 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e27527-06bc-4eb2-91c2-d498ba32205e None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:38.127018 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e27527-06bc-4eb2-91c2-d498ba32205e None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:38.127504 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e27527-06bc-4eb2-91c2-d498ba32205e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:38.148449 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-51e27527-06bc-4eb2-91c2-d498ba32205e None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:07:38.149936 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-51e27527-06bc-4eb2-91c2-d498ba32205e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:07:38.160514 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7f0c97-e5ba-4fc7-aa28-1dd2daf998b7 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:38.173886 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:38.224264 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 148/300] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:07:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:07:38.231830 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e27527-06bc-4eb2-91c2-d498ba32205e None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:38.277584 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7f0c97-e5ba-4fc7-aa28-1dd2daf998b7 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:38.303907 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:38.316693 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b77cfe-9c40-46e5-871c-efb1f0ea068c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:38.319015 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b77cfe-9c40-46e5-871c-efb1f0ea068c None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:38.319461 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b77cfe-9c40-46e5-871c-efb1f0ea068c None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:38.320451 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b77cfe-9c40-46e5-871c-efb1f0ea068c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:38.334435 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b77cfe-9c40-46e5-871c-efb1f0ea068c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:07:38.341108 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-42b77cfe-9c40-46e5-871c-efb1f0ea068c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 17:07:38.341108 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:07:38.341108 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:07:38.341108 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:07:38.341108 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:07:38.341108 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:07:38.341108 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:07:38.341108 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:07:38.341108 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:07:38.341108 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:07:38.341108 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:07:38.341108 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:07:38.341108 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:07:38.341108 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:07:38.341108 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:07:38.341108 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:07:38.342218 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:07:38.342218 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:07:38.342218 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:07:38.342218 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:07:38.342218 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:07:38.342218 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:07:38.342218 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:07:38.342218 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:07:38.342218 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:07:38.342218 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:07:38.342218 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:07:38.342218 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application with Lock( Aug 24 17:07:38.342218 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:07:38.342218 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:07:38.342218 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:07:38.342218 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:07:38.343244 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:07:38.343244 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:07:38.343244 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:07:38.343244 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:07:38.343244 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:07:38.343244 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:07:38.343244 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:07:38.343244 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:07:38.343244 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:07:38.343244 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:07:38.343244 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 17:07:38.343244 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:07:38.350574 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 153/301] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 24 17:07:38 2022] GET /identity/v3/roles/reader => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:07:38.351895 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b77cfe-9c40-46e5-871c-efb1f0ea068c None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:38.363635 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e27527-06bc-4eb2-91c2-d498ba32205e None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:38.441620 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e2c9cb-ff9e-454f-b569-df671ae338ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:38.442542 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e2c9cb-ff9e-454f-b569-df671ae338ab None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:38.443021 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e2c9cb-ff9e-454f-b569-df671ae338ab None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:38.443469 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e2c9cb-ff9e-454f-b569-df671ae338ab None admin] PATH_INFO: `/v3/roles/member` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:38.456686 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e2c9cb-ff9e-454f-b569-df671ae338ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:07:38.462847 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-26e2c9cb-ff9e-454f-b569-df671ae338ab None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:07:38.462847 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:07:38.462847 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:07:38.462847 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:07:38.462847 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:07:38.462847 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:07:38.462847 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:07:38.462847 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:07:38.462847 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:07:38.462847 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:07:38.462847 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:07:38.462847 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:07:38.462847 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:07:38.462847 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:07:38.462847 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:07:38.462847 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:07:38.463735 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:07:38.463735 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:07:38.463735 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:07:38.463735 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:07:38.463735 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:07:38.463735 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:07:38.463735 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:07:38.463735 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:07:38.463735 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:07:38.463735 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:07:38.463735 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:07:38.463735 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 24 17:07:38.463735 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:07:38.463735 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:07:38.463735 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:07:38.463735 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:07:38.464584 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:07:38.464584 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:07:38.464584 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:07:38.464584 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:07:38.464584 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:07:38.464584 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:07:38.464584 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:07:38.464584 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:07:38.464584 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:07:38.464584 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:07:38.464584 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:07:38.464584 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:07:38.467631 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 149/302] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 24 17:07:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:07:38.473217 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e2c9cb-ff9e-454f-b569-df671ae338ab None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:38.494759 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:38.517577 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b77cfe-9c40-46e5-871c-efb1f0ea068c None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:38.561216 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e0f6ff-0a98-495a-abe7-d52a49ae352b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:38.562068 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e0f6ff-0a98-495a-abe7-d52a49ae352b None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:38.562354 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e0f6ff-0a98-495a-abe7-d52a49ae352b None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:38.562628 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e0f6ff-0a98-495a-abe7-d52a49ae352b None admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:38.563465 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e0f6ff-0a98-495a-abe7-d52a49ae352b None admin] RBAC: Adding query filter params (name=reader) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:07:38.563769 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e0f6ff-0a98-495a-abe7-d52a49ae352b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:38.565191 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e0f6ff-0a98-495a-abe7-d52a49ae352b None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:38.578334 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 154/303] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Aug 24 17:07:38 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:38.578334 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e0f6ff-0a98-495a-abe7-d52a49ae352b None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:38.588627 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e2c9cb-ff9e-454f-b569-df671ae338ab None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:38.624749 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ad4956-d297-4e30-b2b7-418bced2d11b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:38.625621 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ad4956-d297-4e30-b2b7-418bced2d11b None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:38.625903 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ad4956-d297-4e30-b2b7-418bced2d11b None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:38.626176 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ad4956-d297-4e30-b2b7-418bced2d11b None admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:38.626841 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ad4956-d297-4e30-b2b7-418bced2d11b None admin] RBAC: Adding query filter params (name=member) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:07:38.627175 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ad4956-d297-4e30-b2b7-418bced2d11b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:38.628546 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ad4956-d297-4e30-b2b7-418bced2d11b None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:38.630455 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:38.641704 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 150/304] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Aug 24 17:07:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:07:38.641704 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ad4956-d297-4e30-b2b7-418bced2d11b None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:38.692301 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e0f6ff-0a98-495a-abe7-d52a49ae352b None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:38.725970 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa1c9bf-9e28-4f1b-920a-6c8ea083f1aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:38.726602 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa1c9bf-9e28-4f1b-920a-6c8ea083f1aa None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:38.726886 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa1c9bf-9e28-4f1b-920a-6c8ea083f1aa None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:38.727182 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa1c9bf-9e28-4f1b-920a-6c8ea083f1aa None admin] PATH_INFO: `/v3/domains/default` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:38.728295 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa1c9bf-9e28-4f1b-920a-6c8ea083f1aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:38.729742 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa1c9bf-9e28-4f1b-920a-6c8ea083f1aa None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:38.731335 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 155/305] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 24 17:07:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:38.732752 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:38.737115 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa1c9bf-9e28-4f1b-920a-6c8ea083f1aa None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:38.758940 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ad4956-d297-4e30-b2b7-418bced2d11b None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:38.796964 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fb6fd8-f884-4bdb-8379-b64f1d0c3119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:38.797599 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fb6fd8-f884-4bdb-8379-b64f1d0c3119 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:38.797879 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fb6fd8-f884-4bdb-8379-b64f1d0c3119 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:38.798152 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fb6fd8-f884-4bdb-8379-b64f1d0c3119 None admin] PATH_INFO: `/v3/users/b41bf8745a9c42f4b0678d0202735596` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:38.813081 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fb6fd8-f884-4bdb-8379-b64f1d0c3119 None admin] RBAC: Authorizing `identity:get_user(user_id=b41bf8745a9c42f4b0678d0202735596)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:38.814548 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fb6fd8-f884-4bdb-8379-b64f1d0c3119 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:38.821222 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 151/306] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 24 17:07:38 2022] GET /identity/v3/users/b41bf8745a9c42f4b0678d0202735596 => generated 293 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:38.821222 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fb6fd8-f884-4bdb-8379-b64f1d0c3119 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:38.845094 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa1c9bf-9e28-4f1b-920a-6c8ea083f1aa None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:38.861250 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:38.884003 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6b63a8-cd9f-4d9f-b53d-3202cc8c4400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:38.884643 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6b63a8-cd9f-4d9f-b53d-3202cc8c4400 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:38.884928 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6b63a8-cd9f-4d9f-b53d-3202cc8c4400 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:38.885201 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6b63a8-cd9f-4d9f-b53d-3202cc8c4400 None admin] PATH_INFO: `/v3/users/c3c3a3eb18c649d696423f05615344a4` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:38.886312 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6b63a8-cd9f-4d9f-b53d-3202cc8c4400 None admin] RBAC: Authorizing `identity:get_user(user_id=c3c3a3eb18c649d696423f05615344a4)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:38.887800 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6b63a8-cd9f-4d9f-b53d-3202cc8c4400 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:38.889411 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 156/307] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 24 17:07:38 2022] GET /identity/v3/users/c3c3a3eb18c649d696423f05615344a4 => generated 293 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:38.889411 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6b63a8-cd9f-4d9f-b53d-3202cc8c4400 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:38.927533 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fb6fd8-f884-4bdb-8379-b64f1d0c3119 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:38.960348 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de687538-5dc4-4a51-a268-43a5225b3d64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:38.960894 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de687538-5dc4-4a51-a268-43a5225b3d64 None admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:38.961176 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de687538-5dc4-4a51-a268-43a5225b3d64 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:38.961451 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de687538-5dc4-4a51-a268-43a5225b3d64 None admin] PATH_INFO: `/v3/groups` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:38.962074 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de687538-5dc4-4a51-a268-43a5225b3d64 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:38.963576 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de687538-5dc4-4a51-a268-43a5225b3d64 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:38.992065 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6b63a8-cd9f-4d9f-b53d-3202cc8c4400 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:39.023563 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:39.023701 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3382fc2-33b5-4140-a285-3876f87f6ab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:39.024187 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3382fc2-33b5-4140-a285-3876f87f6ab1 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:39.024463 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3382fc2-33b5-4140-a285-3876f87f6ab1 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:39.024731 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3382fc2-33b5-4140-a285-3876f87f6ab1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:39.025445 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3382fc2-33b5-4140-a285-3876f87f6ab1 None admin] RBAC: Adding query filter params (role.id=7f65138ab599491194e7f91aa424ceee, user.id=b41bf8745a9c42f4b0678d0202735596, scope.system=all) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:07:39.025746 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3382fc2-33b5-4140-a285-3876f87f6ab1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:39.027180 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3382fc2-33b5-4140-a285-3876f87f6ab1 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:39.028327 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 152/308] 10.222.0.18 () {60 vars in 1134 bytes} [Wed Aug 24 17:07:38 2022] POST /identity/v3/groups => generated 229 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:07:39.031100 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de687538-5dc4-4a51-a268-43a5225b3d64 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:39.032973 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 157/309] 10.222.0.18 () {56 vars in 1308 bytes} [Wed Aug 24 17:07:38 2022] GET /identity/v3/role_assignments?user.id=b41bf8745a9c42f4b0678d0202735596&scope.system=all&role.id=7f65138ab599491194e7f91aa424ceee => generated 226 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:39.034211 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66cb2fe-bc9f-41d8-b1dc-1f0ae974d56d None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:39.038817 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66cb2fe-bc9f-41d8-b1dc-1f0ae974d56d None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:39.038817 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66cb2fe-bc9f-41d8-b1dc-1f0ae974d56d None None] PATH_INFO: `/` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:39.038817 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 158/310] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:07:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:07:39.042194 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8480374b-b7e2-4717-ae2e-34e38f3f3bee None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:39.042512 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8480374b-b7e2-4717-ae2e-34e38f3f3bee None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:39.043235 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8480374b-b7e2-4717-ae2e-34e38f3f3bee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:39.064030 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-8480374b-b7e2-4717-ae2e-34e38f3f3bee None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:07:39.065277 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-8480374b-b7e2-4717-ae2e-34e38f3f3bee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:07:39.128808 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:39.165982 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 159/311] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:07:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:07:39.216091 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de687538-5dc4-4a51-a268-43a5225b3d64 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:39.252064 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1075a574-4fdd-4854-8bf9-ddddb19b27e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:39.253090 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1075a574-4fdd-4854-8bf9-ddddb19b27e1 None admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:39.253564 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1075a574-4fdd-4854-8bf9-ddddb19b27e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:39.254020 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1075a574-4fdd-4854-8bf9-ddddb19b27e1 None admin] PATH_INFO: `/v3/system/users/c3c3a3eb18c649d696423f05615344a4/roles/60dc7e16aee746d5a3e8aa27d7d8525c` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:39.255477 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1075a574-4fdd-4854-8bf9-ddddb19b27e1 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=c3c3a3eb18c649d696423f05615344a4, role_id=60dc7e16aee746d5a3e8aa27d7d8525c)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:39.256852 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1075a574-4fdd-4854-8bf9-ddddb19b27e1 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:39.351634 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 153/312] 10.222.0.18 () {58 vars in 1334 bytes} [Wed Aug 24 17:07:39 2022] PUT /identity/v3/system/users/c3c3a3eb18c649d696423f05615344a4/roles/60dc7e16aee746d5a3e8aa27d7d8525c => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:07:39.447537 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8480374b-b7e2-4717-ae2e-34e38f3f3bee None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:39.565137 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8480374b-b7e2-4717-ae2e-34e38f3f3bee None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:39.601833 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01bcba8-08bf-46fa-b1bf-dd107207cd1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:39.602841 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01bcba8-08bf-46fa-b1bf-dd107207cd1b None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:39.603376 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01bcba8-08bf-46fa-b1bf-dd107207cd1b None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:39.603843 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01bcba8-08bf-46fa-b1bf-dd107207cd1b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:39.615407 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01bcba8-08bf-46fa-b1bf-dd107207cd1b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:07:39.620483 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-a01bcba8-08bf-46fa-b1bf-dd107207cd1b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 17:07:39.620483 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:07:39.620483 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:07:39.620483 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:07:39.620483 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:07:39.620483 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:07:39.620483 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:07:39.620483 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:07:39.620483 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:07:39.620483 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:07:39.620483 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:07:39.620483 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:07:39.620483 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:07:39.620483 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:07:39.620483 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:07:39.620483 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:07:39.621365 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:07:39.621365 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:07:39.621365 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:07:39.621365 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:07:39.621365 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:07:39.621365 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:07:39.621365 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:07:39.621365 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:07:39.621365 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:07:39.621365 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:07:39.621365 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:07:39.621365 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application with Lock( Aug 24 17:07:39.621365 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:07:39.621365 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:07:39.621365 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:07:39.621365 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:07:39.622162 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:07:39.622162 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:07:39.622162 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:07:39.622162 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:07:39.622162 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:07:39.622162 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:07:39.622162 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:07:39.622162 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:07:39.622162 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:07:39.622162 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:07:39.622162 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 17:07:39.622162 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:07:39.624814 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 160/313] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 24 17:07:39 2022] GET /identity/v3/roles/reader => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:07:39.630602 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1075a574-4fdd-4854-8bf9-ddddb19b27e1 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:39.707318 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:39.733085 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1075a574-4fdd-4854-8bf9-ddddb19b27e1 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:39.767733 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5112d61-4150-4c5f-a78d-a504d1d1f7fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:39.768453 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5112d61-4150-4c5f-a78d-a504d1d1f7fd None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:39.768660 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5112d61-4150-4c5f-a78d-a504d1d1f7fd None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:39.768934 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5112d61-4150-4c5f-a78d-a504d1d1f7fd None admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:39.769635 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5112d61-4150-4c5f-a78d-a504d1d1f7fd None admin] RBAC: Adding query filter params (name=reader) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:07:39.775010 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5112d61-4150-4c5f-a78d-a504d1d1f7fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:39.775010 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5112d61-4150-4c5f-a78d-a504d1d1f7fd None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:39.781564 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 154/314] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Aug 24 17:07:39 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:39.787823 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01bcba8-08bf-46fa-b1bf-dd107207cd1b None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:39.826892 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:39.895654 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01bcba8-08bf-46fa-b1bf-dd107207cd1b None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:39.930620 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d220a843-d3ca-4ba9-ab6d-96c25a854023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:39.931603 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d220a843-d3ca-4ba9-ab6d-96c25a854023 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:39.932112 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d220a843-d3ca-4ba9-ab6d-96c25a854023 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:39.932671 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d220a843-d3ca-4ba9-ab6d-96c25a854023 None admin] PATH_INFO: `/v3/users/a4abe9b8215c417a804ce7286737ac6f` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:39.934025 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d220a843-d3ca-4ba9-ab6d-96c25a854023 None admin] RBAC: Authorizing `identity:get_user(user_id=a4abe9b8215c417a804ce7286737ac6f)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:39.935718 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d220a843-d3ca-4ba9-ab6d-96c25a854023 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:39.937836 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 161/315] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 24 17:07:39 2022] GET /identity/v3/users/a4abe9b8215c417a804ce7286737ac6f => generated 297 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:39.943174 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5112d61-4150-4c5f-a78d-a504d1d1f7fd None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:40.045317 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5112d61-4150-4c5f-a78d-a504d1d1f7fd None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:40.069701 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:40.080950 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842705fc-b7c9-49ee-b17b-ee73e2f2b0cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:40.081352 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842705fc-b7c9-49ee-b17b-ee73e2f2b0cd None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:40.081630 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842705fc-b7c9-49ee-b17b-ee73e2f2b0cd None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:40.081899 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842705fc-b7c9-49ee-b17b-ee73e2f2b0cd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:40.158589 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842705fc-b7c9-49ee-b17b-ee73e2f2b0cd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:40.160163 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842705fc-b7c9-49ee-b17b-ee73e2f2b0cd None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:40.267044 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 155/316] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 24 17:07:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 17:07:40.272840 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d220a843-d3ca-4ba9-ab6d-96c25a854023 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:40.372937 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d220a843-d3ca-4ba9-ab6d-96c25a854023 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:40.375189 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:40.406855 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173b58cf-f987-4758-b108-d0e9c5742418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:40.407510 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173b58cf-f987-4758-b108-d0e9c5742418 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:40.407787 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173b58cf-f987-4758-b108-d0e9c5742418 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:40.408056 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173b58cf-f987-4758-b108-d0e9c5742418 None admin] PATH_INFO: `/v3/projects/94586413811643bb999468e84b659403` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:40.409163 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173b58cf-f987-4758-b108-d0e9c5742418 None admin] RBAC: Authorizing `identity:get_project(project_id=94586413811643bb999468e84b659403)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:40.410582 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173b58cf-f987-4758-b108-d0e9c5742418 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:40.412222 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 162/317] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 24 17:07:40 2022] GET /identity/v3/projects/94586413811643bb999468e84b659403 => generated 302 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:40.418134 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842705fc-b7c9-49ee-b17b-ee73e2f2b0cd None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:40.444616 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:40.521302 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842705fc-b7c9-49ee-b17b-ee73e2f2b0cd None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:40.557835 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5578e5-8e1d-4eac-93e0-359f5f2c3f29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:40.557835 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5578e5-8e1d-4eac-93e0-359f5f2c3f29 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:40.558047 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5578e5-8e1d-4eac-93e0-359f5f2c3f29 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:40.558320 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5578e5-8e1d-4eac-93e0-359f5f2c3f29 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:40.559092 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5578e5-8e1d-4eac-93e0-359f5f2c3f29 None admin] RBAC: Adding query filter params (role.id=7f65138ab599491194e7f91aa424ceee, scope.project.id=94586413811643bb999468e84b659403, user.id=a4abe9b8215c417a804ce7286737ac6f) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:07:40.559397 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5578e5-8e1d-4eac-93e0-359f5f2c3f29 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:40.560878 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5578e5-8e1d-4eac-93e0-359f5f2c3f29 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:40.566868 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 156/318] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 24 17:07:40 2022] GET /identity/v3/role_assignments?user.id=a4abe9b8215c417a804ce7286737ac6f&scope.project.id=94586413811643bb999468e84b659403&role.id=7f65138ab599491194e7f91aa424ceee => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:41.068934 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f88755-df90-4ae9-8ad7-b7670e5a8e37 None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:41.069482 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f88755-df90-4ae9-8ad7-b7670e5a8e37 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:41.069482 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f88755-df90-4ae9-8ad7-b7670e5a8e37 None None] PATH_INFO: `/` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:41.070982 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 163/319] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:07:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:07:41.075940 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e0bdfd-7673-4979-86ef-5bd4a95d818d None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:41.076318 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e0bdfd-7673-4979-86ef-5bd4a95d818d None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:41.076475 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e0bdfd-7673-4979-86ef-5bd4a95d818d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:41.089840 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faf510c-151f-475f-8bd5-abffe4122008 None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:41.090086 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faf510c-151f-475f-8bd5-abffe4122008 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:41.090531 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faf510c-151f-475f-8bd5-abffe4122008 None None] PATH_INFO: `/` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:41.092372 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 164/320] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:07:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 17:07:41.095146 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-02e0bdfd-7673-4979-86ef-5bd4a95d818d None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:07:41.096362 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-02e0bdfd-7673-4979-86ef-5bd4a95d818d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:07:41.097166 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04b5d6e-f5a9-4cbb-bd71-2e74ae5b1379 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:41.097448 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04b5d6e-f5a9-4cbb-bd71-2e74ae5b1379 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:41.097718 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04b5d6e-f5a9-4cbb-bd71-2e74ae5b1379 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:41.116547 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-f04b5d6e-f5a9-4cbb-bd71-2e74ae5b1379 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:07:41.117715 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-f04b5d6e-f5a9-4cbb-bd71-2e74ae5b1379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:07:41.178133 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 157/321] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:07:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:07:41.187875 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 165/322] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:07:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 17:07:41.358870 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206c86b3-cf83-442d-a687-372977e7fddb None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:41.359175 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206c86b3-cf83-442d-a687-372977e7fddb None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:41.359377 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206c86b3-cf83-442d-a687-372977e7fddb None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:41.360865 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 158/323] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:07:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:07:41.366102 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c299c238-57af-4555-a069-24b8fad6d5d5 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:41.366398 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c299c238-57af-4555-a069-24b8fad6d5d5 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:41.366680 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c299c238-57af-4555-a069-24b8fad6d5d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:41.385106 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-c299c238-57af-4555-a069-24b8fad6d5d5 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:07:41.386281 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-c299c238-57af-4555-a069-24b8fad6d5d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:07:41.441645 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206c86b3-cf83-442d-a687-372977e7fddb None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:41.454194 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 166/324] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:07:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:07:41.455044 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c299c238-57af-4555-a069-24b8fad6d5d5 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:41.461775 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:41.485160 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:41.564711 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c299c238-57af-4555-a069-24b8fad6d5d5 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:41.570832 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206c86b3-cf83-442d-a687-372977e7fddb None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:41.603780 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6434d311-6e55-4c14-b00d-5559143625ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:41.604667 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6434d311-6e55-4c14-b00d-5559143625ab None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:41.604952 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6434d311-6e55-4c14-b00d-5559143625ab None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:41.605179 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac539469-1ed6-4545-a662-abc3bdea12f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:41.605290 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6434d311-6e55-4c14-b00d-5559143625ab None admin] PATH_INFO: `/v3/domains/default` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:41.605797 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac539469-1ed6-4545-a662-abc3bdea12f2 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:41.606071 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac539469-1ed6-4545-a662-abc3bdea12f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:41.606319 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6434d311-6e55-4c14-b00d-5559143625ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:41.606411 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac539469-1ed6-4545-a662-abc3bdea12f2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:41.607805 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6434d311-6e55-4c14-b00d-5559143625ab None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:41.609584 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 167/325] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 24 17:07:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:41.614851 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6434d311-6e55-4c14-b00d-5559143625ab None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:41.616899 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac539469-1ed6-4545-a662-abc3bdea12f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:07:41.621569 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-ac539469-1ed6-4545-a662-abc3bdea12f2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 17:07:41.621569 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:07:41.621569 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:07:41.621569 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:07:41.621569 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:07:41.621569 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:07:41.621569 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:07:41.621569 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:07:41.621569 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:07:41.621569 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:07:41.621569 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:07:41.621569 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:07:41.621569 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:07:41.621569 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:07:41.621569 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:07:41.621569 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:07:41.622458 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:07:41.622458 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:07:41.622458 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:07:41.622458 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:07:41.622458 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:07:41.622458 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:07:41.622458 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:07:41.622458 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:07:41.622458 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:07:41.622458 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:07:41.622458 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:07:41.622458 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 24 17:07:41.622458 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:07:41.622458 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:07:41.622458 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:07:41.622458 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:07:41.628458 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:07:41.628458 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:07:41.628458 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:07:41.628458 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:07:41.628458 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:07:41.628458 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:07:41.628458 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:07:41.628458 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:07:41.628458 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:07:41.628458 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:07:41.628458 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 17:07:41.628458 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:07:41.628458 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 159/326] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 24 17:07:41 2022] GET /identity/v3/roles/reader => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:07:41.629106 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac539469-1ed6-4545-a662-abc3bdea12f2 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:41.709352 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:41.726973 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:41.729904 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6434d311-6e55-4c14-b00d-5559143625ab None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:41.730988 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac539469-1ed6-4545-a662-abc3bdea12f2 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:41.763999 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758088e8-95e5-418e-9d91-1b8b909f1249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:41.764606 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758088e8-95e5-418e-9d91-1b8b909f1249 None admin] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:41.764890 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758088e8-95e5-418e-9d91-1b8b909f1249 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:41.765168 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758088e8-95e5-418e-9d91-1b8b909f1249 None admin] PATH_INFO: `/v3/groups` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:41.765843 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758088e8-95e5-418e-9d91-1b8b909f1249 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:41.766227 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d7a53d-557b-4c29-842e-660ed6d84318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:41.766828 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d7a53d-557b-4c29-842e-660ed6d84318 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:41.767134 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d7a53d-557b-4c29-842e-660ed6d84318 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:41.767396 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d7a53d-557b-4c29-842e-660ed6d84318 None admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:41.768048 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d7a53d-557b-4c29-842e-660ed6d84318 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:07:41.768166 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758088e8-95e5-418e-9d91-1b8b909f1249 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:41.768345 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d7a53d-557b-4c29-842e-660ed6d84318 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:41.769681 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d7a53d-557b-4c29-842e-660ed6d84318 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:41.777673 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 160/327] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Aug 24 17:07:41 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:07:41.778450 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d7a53d-557b-4c29-842e-660ed6d84318 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:41.869450 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 168/328] 10.222.0.18 () {60 vars in 1134 bytes} [Wed Aug 24 17:07:41 2022] POST /identity/v3/groups => generated 226 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:07:41.870206 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758088e8-95e5-418e-9d91-1b8b909f1249 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:41.878029 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d7a53d-557b-4c29-842e-660ed6d84318 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:41.918819 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08d333e-7357-48ce-9bcc-3c2ac4eb272d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:41.919843 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08d333e-7357-48ce-9bcc-3c2ac4eb272d None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:41.920111 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08d333e-7357-48ce-9bcc-3c2ac4eb272d None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:41.920636 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08d333e-7357-48ce-9bcc-3c2ac4eb272d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:41.929468 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:41.932688 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08d333e-7357-48ce-9bcc-3c2ac4eb272d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:07:41.940874 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-f08d333e-7357-48ce-9bcc-3c2ac4eb272d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:07:41.940874 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:07:41.940874 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:07:41.940874 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:07:41.940874 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:07:41.940874 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:07:41.940874 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:07:41.940874 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:07:41.940874 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:07:41.940874 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:07:41.940874 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:07:41.940874 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:07:41.940874 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:07:41.940874 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:07:41.940874 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:07:41.940874 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:07:41.945417 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:07:41.945417 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:07:41.945417 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:07:41.945417 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:07:41.945417 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:07:41.945417 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:07:41.945417 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:07:41.945417 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:07:41.945417 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:07:41.945417 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:07:41.945417 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:07:41.945417 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 24 17:07:41.945417 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:07:41.945417 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:07:41.945417 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:07:41.945417 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:07:41.946379 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:07:41.946379 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:07:41.946379 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:07:41.946379 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:07:41.946379 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:07:41.946379 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:07:41.946379 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:07:41.946379 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:07:41.946379 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:07:41.946379 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:07:41.946379 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:07:41.946379 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:07:41.946379 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 161/329] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 24 17:07:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:07:41.948086 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08d333e-7357-48ce-9bcc-3c2ac4eb272d None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:41.992697 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758088e8-95e5-418e-9d91-1b8b909f1249 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:42.034891 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8f6bc4-5f7f-450a-9e3f-f2d448c20dc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:42.035577 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8f6bc4-5f7f-450a-9e3f-f2d448c20dc2 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:42.035854 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8f6bc4-5f7f-450a-9e3f-f2d448c20dc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:42.036128 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8f6bc4-5f7f-450a-9e3f-f2d448c20dc2 None admin] PATH_INFO: `/v3/users/b41bf8745a9c42f4b0678d0202735596` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:42.037273 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8f6bc4-5f7f-450a-9e3f-f2d448c20dc2 None admin] RBAC: Authorizing `identity:get_user(user_id=b41bf8745a9c42f4b0678d0202735596)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:42.038719 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8f6bc4-5f7f-450a-9e3f-f2d448c20dc2 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:42.040589 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 169/330] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 24 17:07:41 2022] GET /identity/v3/users/b41bf8745a9c42f4b0678d0202735596 => generated 293 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:42.046858 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8f6bc4-5f7f-450a-9e3f-f2d448c20dc2 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:42.063045 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08d333e-7357-48ce-9bcc-3c2ac4eb272d None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:42.079323 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:42.101853 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de4ab51-af56-48d2-8a97-ca57fe1ca6bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:42.103239 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de4ab51-af56-48d2-8a97-ca57fe1ca6bd None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:42.105292 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de4ab51-af56-48d2-8a97-ca57fe1ca6bd None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:42.105578 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de4ab51-af56-48d2-8a97-ca57fe1ca6bd None admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:42.106319 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de4ab51-af56-48d2-8a97-ca57fe1ca6bd None admin] RBAC: Adding query filter params (name=member) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:07:42.106958 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de4ab51-af56-48d2-8a97-ca57fe1ca6bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:42.108088 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de4ab51-af56-48d2-8a97-ca57fe1ca6bd None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:42.117740 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 162/331] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Aug 24 17:07:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:07:42.125132 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de4ab51-af56-48d2-8a97-ca57fe1ca6bd None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:42.178604 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8f6bc4-5f7f-450a-9e3f-f2d448c20dc2 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:42.205666 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:42.235958 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60321ce3-0c3b-4bb8-9a53-1166a99435bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:42.237298 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60321ce3-0c3b-4bb8-9a53-1166a99435bb None admin] REQUEST_METHOD: `PUT` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:42.237976 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60321ce3-0c3b-4bb8-9a53-1166a99435bb None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:42.238649 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60321ce3-0c3b-4bb8-9a53-1166a99435bb None admin] PATH_INFO: `/v3/system/users/b41bf8745a9c42f4b0678d0202735596/roles/7f65138ab599491194e7f91aa424ceee` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:42.242046 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60321ce3-0c3b-4bb8-9a53-1166a99435bb None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=b41bf8745a9c42f4b0678d0202735596, role_id=7f65138ab599491194e7f91aa424ceee)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:42.245231 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60321ce3-0c3b-4bb8-9a53-1166a99435bb None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:42.260633 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de4ab51-af56-48d2-8a97-ca57fe1ca6bd None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:42.302005 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdc70f2-d571-4f7d-88f7-270c516f5307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:42.302649 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdc70f2-d571-4f7d-88f7-270c516f5307 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:42.302939 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdc70f2-d571-4f7d-88f7-270c516f5307 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:42.303220 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdc70f2-d571-4f7d-88f7-270c516f5307 None admin] PATH_INFO: `/v3/users/c3c3a3eb18c649d696423f05615344a4` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:42.304377 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdc70f2-d571-4f7d-88f7-270c516f5307 None admin] RBAC: Authorizing `identity:get_user(user_id=c3c3a3eb18c649d696423f05615344a4)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:42.305816 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdc70f2-d571-4f7d-88f7-270c516f5307 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:42.307421 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 163/332] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 24 17:07:42 2022] GET /identity/v3/users/c3c3a3eb18c649d696423f05615344a4 => generated 293 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:42.313844 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdc70f2-d571-4f7d-88f7-270c516f5307 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:42.328129 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 170/333] 10.222.0.18 () {58 vars in 1334 bytes} [Wed Aug 24 17:07:42 2022] PUT /identity/v3/system/users/b41bf8745a9c42f4b0678d0202735596/roles/7f65138ab599491194e7f91aa424ceee => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:07:42.437610 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdc70f2-d571-4f7d-88f7-270c516f5307 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:42.470024 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:42.476191 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d5bda3-30ea-4b2e-b814-04de7d518327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:42.476812 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d5bda3-30ea-4b2e-b814-04de7d518327 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:42.477090 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d5bda3-30ea-4b2e-b814-04de7d518327 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:42.477366 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d5bda3-30ea-4b2e-b814-04de7d518327 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:42.478103 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d5bda3-30ea-4b2e-b814-04de7d518327 None admin] RBAC: Adding query filter params (role.id=60dc7e16aee746d5a3e8aa27d7d8525c, user.id=c3c3a3eb18c649d696423f05615344a4, scope.system=all) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:07:42.478412 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d5bda3-30ea-4b2e-b814-04de7d518327 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:42.479914 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d5bda3-30ea-4b2e-b814-04de7d518327 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:42.496207 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 164/334] 10.222.0.18 () {56 vars in 1308 bytes} [Wed Aug 24 17:07:42 2022] GET /identity/v3/role_assignments?user.id=c3c3a3eb18c649d696423f05615344a4&scope.system=all&role.id=60dc7e16aee746d5a3e8aa27d7d8525c => generated 510 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:07:44.084490 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35226a41-744c-44a1-8e78-0b1314979e9f None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:44.085012 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35226a41-744c-44a1-8e78-0b1314979e9f None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:44.085012 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35226a41-744c-44a1-8e78-0b1314979e9f None None] PATH_INFO: `/` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:44.086422 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 171/335] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:07:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:07:44.101406 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98008004-a72e-4a51-baac-b2c56d949cd3 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:44.101928 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98008004-a72e-4a51-baac-b2c56d949cd3 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:44.101928 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98008004-a72e-4a51-baac-b2c56d949cd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:44.123302 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-98008004-a72e-4a51-baac-b2c56d949cd3 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:07:44.124488 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-98008004-a72e-4a51-baac-b2c56d949cd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:07:44.197771 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 165/336] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:07:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:07:44.277301 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d325ea2b-9af6-4762-b17a-c62bca304cf0 None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:44.277598 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d325ea2b-9af6-4762-b17a-c62bca304cf0 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:44.277856 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d325ea2b-9af6-4762-b17a-c62bca304cf0 None None] PATH_INFO: `/` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:44.279277 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 172/337] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:07:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:07:44.287567 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e24bc5-ebf6-4dc5-a1f3-09de925747c0 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:44.287855 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e24bc5-ebf6-4dc5-a1f3-09de925747c0 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:44.288121 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e24bc5-ebf6-4dc5-a1f3-09de925747c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:44.316197 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-79e24bc5-ebf6-4dc5-a1f3-09de925747c0 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:07:44.317399 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-79e24bc5-ebf6-4dc5-a1f3-09de925747c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:07:44.387977 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 166/338] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:07:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:07:44.498231 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08f4d77-d2ce-4042-8299-13a0399c6e03 None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:44.498549 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08f4d77-d2ce-4042-8299-13a0399c6e03 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:44.498790 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08f4d77-d2ce-4042-8299-13a0399c6e03 None None] PATH_INFO: `/` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:44.500698 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 173/339] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:07:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:07:44.505791 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b886a404-77fb-4d19-994b-66a59d0995dd None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:44.506076 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b886a404-77fb-4d19-994b-66a59d0995dd None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:44.506350 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b886a404-77fb-4d19-994b-66a59d0995dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:44.533403 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-b886a404-77fb-4d19-994b-66a59d0995dd None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:07:44.533995 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08f4d77-d2ce-4042-8299-13a0399c6e03 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:44.534597 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-b886a404-77fb-4d19-994b-66a59d0995dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:07:44.631135 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:44.701634 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 167/340] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:07:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:07:44.735274 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:44.749239 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08f4d77-d2ce-4042-8299-13a0399c6e03 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:44.790384 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a81014-06ee-45fe-95d2-46060a74657e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:44.791045 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a81014-06ee-45fe-95d2-46060a74657e None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:44.791326 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a81014-06ee-45fe-95d2-46060a74657e None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:44.791595 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a81014-06ee-45fe-95d2-46060a74657e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:44.805857 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a81014-06ee-45fe-95d2-46060a74657e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:07:44.810685 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-50a81014-06ee-45fe-95d2-46060a74657e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:07:44.810685 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:07:44.810685 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:07:44.810685 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:07:44.810685 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:07:44.810685 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:07:44.810685 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:07:44.810685 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:07:44.810685 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:07:44.810685 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:07:44.810685 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:07:44.810685 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:07:44.810685 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:07:44.810685 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:07:44.810685 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:07:44.810685 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:07:44.811605 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:07:44.811605 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:07:44.811605 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:07:44.811605 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:07:44.811605 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:07:44.811605 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:07:44.811605 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:07:44.811605 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:07:44.811605 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:07:44.811605 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:07:44.811605 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:07:44.811605 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application with Lock( Aug 24 17:07:44.811605 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:07:44.811605 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:07:44.811605 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:07:44.811605 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:07:44.812980 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:07:44.812980 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:07:44.812980 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:07:44.812980 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:07:44.812980 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:07:44.812980 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:07:44.812980 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:07:44.812980 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:07:44.812980 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:07:44.812980 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:07:44.812980 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:07:44.812980 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:07:44.812980 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 174/341] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 24 17:07:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:07:44.817607 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b886a404-77fb-4d19-994b-66a59d0995dd None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:44.867927 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a81014-06ee-45fe-95d2-46060a74657e None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:44.977518 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b886a404-77fb-4d19-994b-66a59d0995dd None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:45.039694 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a81014-06ee-45fe-95d2-46060a74657e None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:45.063919 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803d4060-0552-43a6-84f8-a52fc4f341f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:45.064515 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803d4060-0552-43a6-84f8-a52fc4f341f2 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:45.064855 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803d4060-0552-43a6-84f8-a52fc4f341f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:45.065136 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803d4060-0552-43a6-84f8-a52fc4f341f2 None admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:45.065817 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803d4060-0552-43a6-84f8-a52fc4f341f2 None admin] RBAC: Adding query filter params (name=member) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:07:45.066116 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803d4060-0552-43a6-84f8-a52fc4f341f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:45.067628 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803d4060-0552-43a6-84f8-a52fc4f341f2 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:45.085419 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 168/342] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Aug 24 17:07:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:45.086273 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803d4060-0552-43a6-84f8-a52fc4f341f2 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:45.093073 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58b4f44-f5da-4d1a-91f8-feedfc9cb2ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:45.094395 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58b4f44-f5da-4d1a-91f8-feedfc9cb2ec None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:45.094877 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58b4f44-f5da-4d1a-91f8-feedfc9cb2ec None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:45.095373 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58b4f44-f5da-4d1a-91f8-feedfc9cb2ec None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:45.112933 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58b4f44-f5da-4d1a-91f8-feedfc9cb2ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:07:45.118072 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-a58b4f44-f5da-4d1a-91f8-feedfc9cb2ec None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 17:07:45.118072 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:07:45.118072 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:07:45.118072 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:07:45.118072 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:07:45.118072 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:07:45.118072 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:07:45.118072 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:07:45.118072 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:07:45.118072 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:07:45.118072 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:07:45.118072 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:07:45.118072 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:07:45.118072 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:07:45.118072 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:07:45.118072 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:07:45.118990 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:07:45.118990 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:07:45.118990 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:07:45.118990 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:07:45.118990 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:07:45.118990 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:07:45.118990 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:07:45.118990 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:07:45.118990 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:07:45.118990 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:07:45.118990 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:07:45.118990 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application with Lock( Aug 24 17:07:45.118990 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:07:45.118990 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:07:45.118990 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:07:45.118990 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:07:45.119794 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:07:45.119794 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:07:45.119794 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:07:45.119794 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:07:45.119794 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:07:45.119794 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:07:45.119794 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:07:45.119794 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:07:45.119794 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:07:45.119794 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:07:45.119794 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 17:07:45.119794 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:07:45.127170 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 175/343] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Aug 24 17:07:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:07:45.128210 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58b4f44-f5da-4d1a-91f8-feedfc9cb2ec None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:45.155245 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:45.156045 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:45.234517 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803d4060-0552-43a6-84f8-a52fc4f341f2 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:45.263970 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58b4f44-f5da-4d1a-91f8-feedfc9cb2ec None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:45.286673 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d785ffc3-a91d-4f59-a46c-20ad5035f648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:45.287349 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d785ffc3-a91d-4f59-a46c-20ad5035f648 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:45.287729 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d785ffc3-a91d-4f59-a46c-20ad5035f648 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:45.288012 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d785ffc3-a91d-4f59-a46c-20ad5035f648 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:45.302707 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d785ffc3-a91d-4f59-a46c-20ad5035f648 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:07:45.304027 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafeabc1-bd8b-4659-9049-776cd94e5b39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:45.304836 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafeabc1-bd8b-4659-9049-776cd94e5b39 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:45.305293 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafeabc1-bd8b-4659-9049-776cd94e5b39 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:45.305764 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafeabc1-bd8b-4659-9049-776cd94e5b39 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:45.307943 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-d785ffc3-a91d-4f59-a46c-20ad5035f648 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 17:07:45.307943 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:07:45.307943 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:07:45.307943 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:07:45.307943 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:07:45.307943 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:07:45.307943 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:07:45.307943 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:07:45.307943 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:07:45.307943 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:07:45.307943 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:07:45.307943 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:07:45.307943 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:07:45.307943 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:07:45.307943 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:07:45.307943 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:07:45.308835 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:07:45.308835 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:07:45.308835 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:07:45.308835 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:07:45.308835 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:07:45.308835 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:07:45.308835 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:07:45.308835 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:07:45.308835 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:07:45.308835 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:07:45.308835 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:07:45.308835 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 24 17:07:45.308835 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:07:45.308835 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:07:45.308835 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:07:45.308835 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:07:45.310989 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:07:45.310989 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:07:45.310989 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:07:45.310989 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:07:45.310989 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:07:45.310989 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:07:45.310989 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:07:45.310989 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:07:45.310989 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:07:45.310989 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:07:45.310989 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 17:07:45.310989 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:07:45.310989 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 169/344] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 24 17:07:45 2022] GET /identity/v3/roles/reader => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:07:45.310989 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d785ffc3-a91d-4f59-a46c-20ad5035f648 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:45.396686 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafeabc1-bd8b-4659-9049-776cd94e5b39 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:45.398575 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafeabc1-bd8b-4659-9049-776cd94e5b39 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:45.416518 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:45.447847 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:45.450778 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d785ffc3-a91d-4f59-a46c-20ad5035f648 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:45.495095 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ce4b7f-9864-484c-bc5d-49c44a0c6e7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:45.496276 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ce4b7f-9864-484c-bc5d-49c44a0c6e7e None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:45.496276 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ce4b7f-9864-484c-bc5d-49c44a0c6e7e None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:45.496276 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ce4b7f-9864-484c-bc5d-49c44a0c6e7e None admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:45.497089 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ce4b7f-9864-484c-bc5d-49c44a0c6e7e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:07:45.497457 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ce4b7f-9864-484c-bc5d-49c44a0c6e7e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:45.500099 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ce4b7f-9864-484c-bc5d-49c44a0c6e7e None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:45.514584 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 170/345] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 24 17:07:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:45.514584 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d49aea-5a22-4e39-908a-bba7d9e281ab None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:45.514584 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d49aea-5a22-4e39-908a-bba7d9e281ab None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:45.514584 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d49aea-5a22-4e39-908a-bba7d9e281ab None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:45.514584 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 171/346] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:07:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:07:45.515429 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d49aea-5a22-4e39-908a-bba7d9e281ab None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:45.536740 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 176/347] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 24 17:07:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 17:07:45.538480 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadb13f6-7934-4c94-98b2-d4ab10399f32 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:45.539058 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadb13f6-7934-4c94-98b2-d4ab10399f32 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:45.539681 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadb13f6-7934-4c94-98b2-d4ab10399f32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:45.564481 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-aadb13f6-7934-4c94-98b2-d4ab10399f32 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:07:45.566230 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-aadb13f6-7934-4c94-98b2-d4ab10399f32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:07:45.654002 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 177/348] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:07:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:07:45.655224 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadb13f6-7934-4c94-98b2-d4ab10399f32 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:45.662892 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d49aea-5a22-4e39-908a-bba7d9e281ab None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:45.705477 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:45.711956 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfbdc08-95d9-4dd9-b56a-3826c8c54891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:45.716411 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfbdc08-95d9-4dd9-b56a-3826c8c54891 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:45.716411 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfbdc08-95d9-4dd9-b56a-3826c8c54891 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:45.716411 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfbdc08-95d9-4dd9-b56a-3826c8c54891 None admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:45.716411 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfbdc08-95d9-4dd9-b56a-3826c8c54891 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:07:45.716411 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfbdc08-95d9-4dd9-b56a-3826c8c54891 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:45.716803 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfbdc08-95d9-4dd9-b56a-3826c8c54891 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:45.730616 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 172/349] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Aug 24 17:07:45 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:45.731911 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfbdc08-95d9-4dd9-b56a-3826c8c54891 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:45.767669 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:45.776730 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadb13f6-7934-4c94-98b2-d4ab10399f32 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:45.816164 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e24eda5-15fd-4c60-89b8-3570bd3f13df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:45.817152 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e24eda5-15fd-4c60-89b8-3570bd3f13df None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:45.817643 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e24eda5-15fd-4c60-89b8-3570bd3f13df None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:45.818108 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e24eda5-15fd-4c60-89b8-3570bd3f13df None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:45.867608 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfbdc08-95d9-4dd9-b56a-3826c8c54891 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:45.900776 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e24eda5-15fd-4c60-89b8-3570bd3f13df None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:45.902572 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e24eda5-15fd-4c60-89b8-3570bd3f13df None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:45.914931 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff091374-d827-419e-8580-918a622a4ec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:45.915828 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff091374-d827-419e-8580-918a622a4ec0 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:45.916111 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff091374-d827-419e-8580-918a622a4ec0 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:45.916388 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff091374-d827-419e-8580-918a622a4ec0 None admin] PATH_INFO: `/v3/projects/9161a24523d545a8939fb56c3c930122` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:45.917792 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff091374-d827-419e-8580-918a622a4ec0 None admin] RBAC: Authorizing `identity:get_project(project_id=9161a24523d545a8939fb56c3c930122)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:45.920265 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff091374-d827-419e-8580-918a622a4ec0 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:45.922210 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 173/350] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 24 17:07:45 2022] GET /identity/v3/projects/9161a24523d545a8939fb56c3c930122 => generated 298 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:45.926602 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff091374-d827-419e-8580-918a622a4ec0 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:45.962273 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:46.029033 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 178/351] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 24 17:07:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 17:07:46.029900 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e24eda5-15fd-4c60-89b8-3570bd3f13df None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:46.043896 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:46.056341 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff091374-d827-419e-8580-918a622a4ec0 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:46.104095 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea25a4b-31ea-443b-9780-75fdac2d620e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:46.104735 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea25a4b-31ea-443b-9780-75fdac2d620e None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:46.105016 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea25a4b-31ea-443b-9780-75fdac2d620e None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:46.105292 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea25a4b-31ea-443b-9780-75fdac2d620e None admin] PATH_INFO: `/v3/users/b41bf8745a9c42f4b0678d0202735596` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:46.106662 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea25a4b-31ea-443b-9780-75fdac2d620e None admin] RBAC: Authorizing `identity:get_user(user_id=b41bf8745a9c42f4b0678d0202735596)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:46.108820 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea25a4b-31ea-443b-9780-75fdac2d620e None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:46.110747 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 174/352] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 24 17:07:45 2022] GET /identity/v3/users/b41bf8745a9c42f4b0678d0202735596 => generated 293 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:46.111588 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea25a4b-31ea-443b-9780-75fdac2d620e None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:46.168333 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e24eda5-15fd-4c60-89b8-3570bd3f13df None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:46.220494 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e02519f-1026-4ffd-8918-9e000b42e98f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:46.221133 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e02519f-1026-4ffd-8918-9e000b42e98f None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:46.221413 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e02519f-1026-4ffd-8918-9e000b42e98f None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:46.221688 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e02519f-1026-4ffd-8918-9e000b42e98f None admin] PATH_INFO: `/v3/groups/3dad4633ec274cfb81918f0a9e56bead` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:46.230506 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e02519f-1026-4ffd-8918-9e000b42e98f None admin] RBAC: Authorizing `identity:get_group(group_id=3dad4633ec274cfb81918f0a9e56bead)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:46.232707 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e02519f-1026-4ffd-8918-9e000b42e98f None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:46.234659 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 179/353] 10.222.0.18 () {56 vars in 1180 bytes} [Wed Aug 24 17:07:46 2022] GET /identity/v3/groups/3dad4633ec274cfb81918f0a9e56bead => generated 226 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:46.235469 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e02519f-1026-4ffd-8918-9e000b42e98f None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:46.251274 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:46.270854 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea25a4b-31ea-443b-9780-75fdac2d620e None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:46.308442 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05f3a44-34ff-474a-bb20-56a190fefabb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:46.309058 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05f3a44-34ff-474a-bb20-56a190fefabb None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:46.309338 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05f3a44-34ff-474a-bb20-56a190fefabb None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:46.309615 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05f3a44-34ff-474a-bb20-56a190fefabb None admin] PATH_INFO: `/v3/projects/9161a24523d545a8939fb56c3c930122` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:46.310923 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05f3a44-34ff-474a-bb20-56a190fefabb None admin] RBAC: Authorizing `identity:get_project(project_id=9161a24523d545a8939fb56c3c930122)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:46.313471 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05f3a44-34ff-474a-bb20-56a190fefabb None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:46.318196 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 175/354] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 24 17:07:46 2022] GET /identity/v3/projects/9161a24523d545a8939fb56c3c930122 => generated 298 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:46.319006 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05f3a44-34ff-474a-bb20-56a190fefabb None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:46.349659 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:46.351304 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e02519f-1026-4ffd-8918-9e000b42e98f None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:46.391520 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70adb54a-1fe3-4636-b7b9-28d249e5bf05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:46.392140 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70adb54a-1fe3-4636-b7b9-28d249e5bf05 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:46.392421 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70adb54a-1fe3-4636-b7b9-28d249e5bf05 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:46.392693 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70adb54a-1fe3-4636-b7b9-28d249e5bf05 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:46.393727 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70adb54a-1fe3-4636-b7b9-28d249e5bf05 None admin] RBAC: Adding query filter params (role.id=7f65138ab599491194e7f91aa424ceee, user.id=b41bf8745a9c42f4b0678d0202735596, scope.system=all) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:07:46.394134 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70adb54a-1fe3-4636-b7b9-28d249e5bf05 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:46.396166 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70adb54a-1fe3-4636-b7b9-28d249e5bf05 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:46.412107 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 180/355] 10.222.0.18 () {56 vars in 1308 bytes} [Wed Aug 24 17:07:46 2022] GET /identity/v3/role_assignments?user.id=b41bf8745a9c42f4b0678d0202735596&scope.system=all&role.id=7f65138ab599491194e7f91aa424ceee => generated 510 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:46.415168 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70adb54a-1fe3-4636-b7b9-28d249e5bf05 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:46.483609 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05f3a44-34ff-474a-bb20-56a190fefabb None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:46.530339 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedb791f-279b-49cb-9086-44fad144a882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:46.531472 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedb791f-279b-49cb-9086-44fad144a882 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:46.531951 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedb791f-279b-49cb-9086-44fad144a882 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:46.532477 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedb791f-279b-49cb-9086-44fad144a882 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:46.551546 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedb791f-279b-49cb-9086-44fad144a882 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:07:46.559299 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-aedb791f-279b-49cb-9086-44fad144a882 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:07:46.559299 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:07:46.559299 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:07:46.559299 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:07:46.559299 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:07:46.559299 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:07:46.559299 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:07:46.559299 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:07:46.559299 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:07:46.559299 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:07:46.559299 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:07:46.559299 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:07:46.559299 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:07:46.559299 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:07:46.559299 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:07:46.559299 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:07:46.560240 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:07:46.560240 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:07:46.560240 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:07:46.560240 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:07:46.560240 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:07:46.560240 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:07:46.560240 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:07:46.560240 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:07:46.560240 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:07:46.560240 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:07:46.560240 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:07:46.560240 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 24 17:07:46.560240 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:07:46.560240 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:07:46.560240 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:07:46.560240 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:07:46.561065 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:07:46.561065 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:07:46.561065 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:07:46.561065 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:07:46.561065 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:07:46.561065 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:07:46.561065 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:07:46.561065 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:07:46.561065 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:07:46.561065 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:07:46.561065 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:07:46.561065 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:07:46.563557 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 176/356] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 24 17:07:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:07:46.564652 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedb791f-279b-49cb-9086-44fad144a882 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:46.585690 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:46.660210 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70adb54a-1fe3-4636-b7b9-28d249e5bf05 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:46.677478 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:46.692714 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedb791f-279b-49cb-9086-44fad144a882 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:46.705309 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f738e3-18b2-44ea-afa3-51f014a3c41b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:46.706247 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f738e3-18b2-44ea-afa3-51f014a3c41b None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:46.706738 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f738e3-18b2-44ea-afa3-51f014a3c41b None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:46.707265 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f738e3-18b2-44ea-afa3-51f014a3c41b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:46.708198 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f738e3-18b2-44ea-afa3-51f014a3c41b None admin] RBAC: Adding query filter params (group.id=3dad4633ec274cfb81918f0a9e56bead, role.id=60dc7e16aee746d5a3e8aa27d7d8525c, scope.project.id=9161a24523d545a8939fb56c3c930122) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:07:46.708640 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f738e3-18b2-44ea-afa3-51f014a3c41b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:46.710220 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f738e3-18b2-44ea-afa3-51f014a3c41b None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:46.719230 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 181/357] 10.222.0.18 () {56 vars in 1376 bytes} [Wed Aug 24 17:07:46 2022] GET /identity/v3/role_assignments?group.id=3dad4633ec274cfb81918f0a9e56bead&scope.project.id=9161a24523d545a8939fb56c3c930122&role.id=60dc7e16aee746d5a3e8aa27d7d8525c => generated 260 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:46.723232 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f738e3-18b2-44ea-afa3-51f014a3c41b None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:46.745087 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4da64b8-6133-4550-b1d8-105b74c74744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:46.746004 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4da64b8-6133-4550-b1d8-105b74c74744 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:46.747325 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4da64b8-6133-4550-b1d8-105b74c74744 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:46.747746 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4da64b8-6133-4550-b1d8-105b74c74744 None admin] PATH_INFO: `/v3/groups/3dad4633ec274cfb81918f0a9e56bead` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:46.750168 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4da64b8-6133-4550-b1d8-105b74c74744 None admin] RBAC: Authorizing `identity:get_group(group_id=3dad4633ec274cfb81918f0a9e56bead)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:46.752102 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4da64b8-6133-4550-b1d8-105b74c74744 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:46.754561 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 177/358] 10.222.0.18 () {56 vars in 1180 bytes} [Wed Aug 24 17:07:46 2022] GET /identity/v3/groups/3dad4633ec274cfb81918f0a9e56bead => generated 226 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:46.756296 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46c6b65-b978-4baa-9c94-4d3f5abd0009 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:46.756678 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46c6b65-b978-4baa-9c94-4d3f5abd0009 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:46.757306 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46c6b65-b978-4baa-9c94-4d3f5abd0009 None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:46.759669 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 178/359] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:07:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:07:46.760530 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46c6b65-b978-4baa-9c94-4d3f5abd0009 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:46.857998 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f738e3-18b2-44ea-afa3-51f014a3c41b None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:46.895943 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:46.908402 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46c6b65-b978-4baa-9c94-4d3f5abd0009 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:46.911719 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3f441b-b035-4113-97db-b45da504d4dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:46.912394 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3f441b-b035-4113-97db-b45da504d4dd None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:46.912675 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3f441b-b035-4113-97db-b45da504d4dd None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:46.912943 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3f441b-b035-4113-97db-b45da504d4dd None admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:46.913608 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3f441b-b035-4113-97db-b45da504d4dd None admin] RBAC: Adding query filter params (name=member) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:07:46.913912 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3f441b-b035-4113-97db-b45da504d4dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:46.915310 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3f441b-b035-4113-97db-b45da504d4dd None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:46.924800 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 182/360] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Aug 24 17:07:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:46.930381 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a543155-eeef-416b-b47b-788a275d0f91 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:46.930670 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a543155-eeef-416b-b47b-788a275d0f91 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:46.930962 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a543155-eeef-416b-b47b-788a275d0f91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:46.950782 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-6a543155-eeef-416b-b47b-788a275d0f91 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:07:46.952010 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-6a543155-eeef-416b-b47b-788a275d0f91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:07:46.952222 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a861c4d-b335-4f40-b73c-b8fefe64c617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:46.952999 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a861c4d-b335-4f40-b73c-b8fefe64c617 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:46.953413 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a861c4d-b335-4f40-b73c-b8fefe64c617 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:46.953936 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a861c4d-b335-4f40-b73c-b8fefe64c617 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:46.956072 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a861c4d-b335-4f40-b73c-b8fefe64c617 None admin] RBAC: Adding query filter params (group.id=3dad4633ec274cfb81918f0a9e56bead, role.id=af19a7a2fae5422da5bd7daea4bbfccf, scope.project.id=9161a24523d545a8939fb56c3c930122) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:07:46.956072 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a861c4d-b335-4f40-b73c-b8fefe64c617 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:46.956659 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a861c4d-b335-4f40-b73c-b8fefe64c617 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:46.966946 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 179/361] 10.222.0.18 () {56 vars in 1376 bytes} [Wed Aug 24 17:07:46 2022] GET /identity/v3/role_assignments?group.id=3dad4633ec274cfb81918f0a9e56bead&scope.project.id=9161a24523d545a8939fb56c3c930122&role.id=af19a7a2fae5422da5bd7daea4bbfccf => generated 260 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:46.967796 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3799eba-3b5a-42d7-bcf5-c83f34927cec None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:46.968026 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3799eba-3b5a-42d7-bcf5-c83f34927cec None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:46.968294 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3799eba-3b5a-42d7-bcf5-c83f34927cec None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:46.969798 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 180/362] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:07:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:07:46.970382 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3799eba-3b5a-42d7-bcf5-c83f34927cec None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:47.004898 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:47.058523 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 183/363] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:07:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:07:47.060603 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ed9ed6-693d-4f9f-a609-433321061e29 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:47.060886 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ed9ed6-693d-4f9f-a609-433321061e29 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:47.061309 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ed9ed6-693d-4f9f-a609-433321061e29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:47.090297 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-d1ed9ed6-693d-4f9f-a609-433321061e29 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:07:47.092086 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-d1ed9ed6-693d-4f9f-a609-433321061e29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:07:47.147731 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3799eba-3b5a-42d7-bcf5-c83f34927cec None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:47.188954 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 184/364] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:07:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:07:47.208237 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cc0e82-dfe3-45fa-9c4e-1465319677f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:47.208963 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cc0e82-dfe3-45fa-9c4e-1465319677f8 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:47.209287 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cc0e82-dfe3-45fa-9c4e-1465319677f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:47.209615 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cc0e82-dfe3-45fa-9c4e-1465319677f8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:47.283434 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cc0e82-dfe3-45fa-9c4e-1465319677f8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:47.284899 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cc0e82-dfe3-45fa-9c4e-1465319677f8 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:47.306134 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:47.398425 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 181/365] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 24 17:07:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 17:07:47.404757 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ed9ed6-693d-4f9f-a609-433321061e29 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:47.456559 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:47.468735 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cc0e82-dfe3-45fa-9c4e-1465319677f8 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:47.529314 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ed9ed6-693d-4f9f-a609-433321061e29 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:47.570174 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7b081e-5ab5-4ce4-b962-fab6cf778de2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:47.570812 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7b081e-5ab5-4ce4-b962-fab6cf778de2 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:47.571134 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7b081e-5ab5-4ce4-b962-fab6cf778de2 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:47.571408 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7b081e-5ab5-4ce4-b962-fab6cf778de2 None admin] PATH_INFO: `/v3/projects/94586413811643bb999468e84b659403` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:47.572541 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7b081e-5ab5-4ce4-b962-fab6cf778de2 None admin] RBAC: Authorizing `identity:get_project(project_id=94586413811643bb999468e84b659403)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:47.574036 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7b081e-5ab5-4ce4-b962-fab6cf778de2 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:47.576568 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 185/366] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 24 17:07:47 2022] GET /identity/v3/projects/94586413811643bb999468e84b659403 => generated 302 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:07:47.585439 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7b081e-5ab5-4ce4-b962-fab6cf778de2 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:47.675186 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cc0e82-dfe3-45fa-9c4e-1465319677f8 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:47.697213 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:47.713855 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3cc18d-1872-4929-a16f-a18e74207e8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:47.714487 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3cc18d-1872-4929-a16f-a18e74207e8a None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:47.714774 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3cc18d-1872-4929-a16f-a18e74207e8a None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:47.715058 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3cc18d-1872-4929-a16f-a18e74207e8a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:47.715219 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7b081e-5ab5-4ce4-b962-fab6cf778de2 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:47.726612 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3cc18d-1872-4929-a16f-a18e74207e8a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:07:47.731691 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-ff3cc18d-1872-4929-a16f-a18e74207e8a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 17:07:47.731691 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:07:47.731691 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:07:47.731691 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:07:47.731691 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:07:47.731691 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:07:47.731691 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:07:47.731691 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:07:47.731691 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:07:47.731691 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:07:47.731691 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:07:47.731691 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:07:47.731691 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:07:47.731691 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:07:47.731691 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:07:47.731691 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:07:47.732597 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:07:47.732597 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:07:47.732597 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:07:47.732597 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:07:47.732597 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:07:47.732597 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:07:47.732597 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:07:47.732597 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:07:47.732597 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:07:47.732597 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:07:47.732597 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:07:47.732597 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 24 17:07:47.732597 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:07:47.732597 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:07:47.732597 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:07:47.732597 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:07:47.733383 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:07:47.733383 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:07:47.733383 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:07:47.733383 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:07:47.733383 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:07:47.733383 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:07:47.733383 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:07:47.733383 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:07:47.733383 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:07:47.733383 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:07:47.733383 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 17:07:47.733383 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:07:47.733383 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 182/367] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Aug 24 17:07:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:07:47.734041 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3cc18d-1872-4929-a16f-a18e74207e8a None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:47.739601 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:47.755131 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebd75ff-056e-46ae-b74b-850b721a82c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:47.755780 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebd75ff-056e-46ae-b74b-850b721a82c4 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:47.756118 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebd75ff-056e-46ae-b74b-850b721a82c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:47.756427 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebd75ff-056e-46ae-b74b-850b721a82c4 None admin] PATH_INFO: `/v3/groups/3dad4633ec274cfb81918f0a9e56bead` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:47.758170 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebd75ff-056e-46ae-b74b-850b721a82c4 None admin] RBAC: Authorizing `identity:get_group(group_id=3dad4633ec274cfb81918f0a9e56bead)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:47.759755 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebd75ff-056e-46ae-b74b-850b721a82c4 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:47.761589 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 186/368] 10.222.0.18 () {56 vars in 1180 bytes} [Wed Aug 24 17:07:47 2022] GET /identity/v3/groups/3dad4633ec274cfb81918f0a9e56bead => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:07:47.762366 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebd75ff-056e-46ae-b74b-850b721a82c4 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:47.854345 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3cc18d-1872-4929-a16f-a18e74207e8a None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:47.894682 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebd75ff-056e-46ae-b74b-850b721a82c4 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:47.897490 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50502462-237f-4d52-8769-8dbec117b1fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:47.898135 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50502462-237f-4d52-8769-8dbec117b1fb None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:47.898417 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50502462-237f-4d52-8769-8dbec117b1fb None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:47.898687 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50502462-237f-4d52-8769-8dbec117b1fb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:47.912383 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50502462-237f-4d52-8769-8dbec117b1fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:07:47.917508 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-50502462-237f-4d52-8769-8dbec117b1fb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:07:47.917508 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:07:47.917508 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:07:47.917508 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:07:47.917508 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:07:47.917508 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:07:47.917508 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:07:47.917508 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:07:47.917508 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:07:47.917508 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:07:47.917508 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:07:47.917508 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:07:47.917508 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:07:47.917508 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:07:47.917508 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:07:47.917508 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:07:47.918397 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:07:47.918397 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:07:47.918397 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:07:47.918397 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:07:47.918397 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:07:47.918397 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:07:47.918397 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:07:47.918397 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:07:47.918397 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:07:47.918397 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:07:47.918397 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:07:47.918397 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 24 17:07:47.918397 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:07:47.918397 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:07:47.918397 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:07:47.918397 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:07:47.919221 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:07:47.919221 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:07:47.919221 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:07:47.919221 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:07:47.919221 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:07:47.919221 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:07:47.919221 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:07:47.919221 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:07:47.919221 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:07:47.919221 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:07:47.919221 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:07:47.919221 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:07:47.923414 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 183/369] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Aug 24 17:07:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:07:47.924290 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50502462-237f-4d52-8769-8dbec117b1fb None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:47.936061 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d5754b-ed82-4466-adde-dd58672073a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:47.937004 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d5754b-ed82-4466-adde-dd58672073a0 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:47.937506 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d5754b-ed82-4466-adde-dd58672073a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:47.937965 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d5754b-ed82-4466-adde-dd58672073a0 None admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:47.938958 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d5754b-ed82-4466-adde-dd58672073a0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:07:47.939453 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d5754b-ed82-4466-adde-dd58672073a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:47.941099 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d5754b-ed82-4466-adde-dd58672073a0 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:47.950325 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 187/370] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 24 17:07:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:47.952812 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d5754b-ed82-4466-adde-dd58672073a0 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:47.976693 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:48.010070 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:48.057398 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50502462-237f-4d52-8769-8dbec117b1fb None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:48.063820 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d5754b-ed82-4466-adde-dd58672073a0 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:48.104758 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2084690-cfce-4830-b670-a99b0f0cb7f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:48.105394 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2084690-cfce-4830-b670-a99b0f0cb7f8 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:48.105672 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2084690-cfce-4830-b670-a99b0f0cb7f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:48.105946 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2084690-cfce-4830-b670-a99b0f0cb7f8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:48.106808 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2084690-cfce-4830-b670-a99b0f0cb7f8 None admin] RBAC: Adding query filter params (group.id=3dad4633ec274cfb81918f0a9e56bead, role.id=60dc7e16aee746d5a3e8aa27d7d8525c, scope.project.id=94586413811643bb999468e84b659403) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:07:48.107698 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e04132-2c8d-457c-a923-434f5ef8b14d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:48.107813 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2084690-cfce-4830-b670-a99b0f0cb7f8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:48.108610 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e04132-2c8d-457c-a923-434f5ef8b14d None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:48.109077 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e04132-2c8d-457c-a923-434f5ef8b14d None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:48.109245 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2084690-cfce-4830-b670-a99b0f0cb7f8 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:48.109697 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e04132-2c8d-457c-a923-434f5ef8b14d None admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:48.110544 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e04132-2c8d-457c-a923-434f5ef8b14d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:07:48.111078 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e04132-2c8d-457c-a923-434f5ef8b14d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:48.112545 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e04132-2c8d-457c-a923-434f5ef8b14d None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:48.115503 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 184/371] 10.222.0.18 () {56 vars in 1376 bytes} [Wed Aug 24 17:07:47 2022] GET /identity/v3/role_assignments?group.id=3dad4633ec274cfb81918f0a9e56bead&scope.project.id=94586413811643bb999468e84b659403&role.id=60dc7e16aee746d5a3e8aa27d7d8525c => generated 260 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:48.116672 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2084690-cfce-4830-b670-a99b0f0cb7f8 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:48.139247 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 188/372] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 24 17:07:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:48.146253 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e04132-2c8d-457c-a923-434f5ef8b14d None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:48.283934 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2084690-cfce-4830-b670-a99b0f0cb7f8 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:48.288992 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:48.354055 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457b176d-e285-417f-8b80-a6fcde05791f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:48.354780 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457b176d-e285-417f-8b80-a6fcde05791f None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:48.355183 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457b176d-e285-417f-8b80-a6fcde05791f None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:48.355353 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457b176d-e285-417f-8b80-a6fcde05791f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:48.362309 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e04132-2c8d-457c-a923-434f5ef8b14d None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:48.398391 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:48.406915 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d422021e-da68-400f-aa00-40683b9657ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:48.407844 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d422021e-da68-400f-aa00-40683b9657ab None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:48.408542 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d422021e-da68-400f-aa00-40683b9657ab None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:48.409019 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d422021e-da68-400f-aa00-40683b9657ab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:48.437622 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457b176d-e285-417f-8b80-a6fcde05791f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:48.439129 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457b176d-e285-417f-8b80-a6fcde05791f None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:48.485502 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d422021e-da68-400f-aa00-40683b9657ab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:48.487383 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d422021e-da68-400f-aa00-40683b9657ab None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:48.601256 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 189/373] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 24 17:07:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 17:07:48.607963 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d422021e-da68-400f-aa00-40683b9657ab None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:48.620440 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 185/374] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 24 17:07:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 504 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 17:07:48.626993 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457b176d-e285-417f-8b80-a6fcde05791f None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:48.630679 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:48.634698 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:48.786497 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d422021e-da68-400f-aa00-40683b9657ab None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:48.799448 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457b176d-e285-417f-8b80-a6fcde05791f None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:48.843702 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa1e6db-ba84-4ea5-82d4-b73dbc18b6db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:48.844762 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa1e6db-ba84-4ea5-82d4-b73dbc18b6db None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:48.845295 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa1e6db-ba84-4ea5-82d4-b73dbc18b6db None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:48.845716 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa1e6db-ba84-4ea5-82d4-b73dbc18b6db None admin] PATH_INFO: `/v3/projects/94586413811643bb999468e84b659403` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:48.847244 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa1e6db-ba84-4ea5-82d4-b73dbc18b6db None admin] RBAC: Authorizing `identity:get_project(project_id=94586413811643bb999468e84b659403)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:48.848844 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa1e6db-ba84-4ea5-82d4-b73dbc18b6db None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:48.850980 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 186/375] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 24 17:07:48 2022] GET /identity/v3/projects/94586413811643bb999468e84b659403 => generated 302 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:07:48.856397 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa1e6db-ba84-4ea5-82d4-b73dbc18b6db None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:48.897485 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d521e72-c26d-4e79-b82d-a3427159a631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:48.898137 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d521e72-c26d-4e79-b82d-a3427159a631 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:48.898417 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d521e72-c26d-4e79-b82d-a3427159a631 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:48.898690 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d521e72-c26d-4e79-b82d-a3427159a631 None admin] PATH_INFO: `/v3/projects/68a3eec6caa5424ea6bb39d5d6e46c2f` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:48.903606 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d521e72-c26d-4e79-b82d-a3427159a631 None admin] RBAC: Authorizing `identity:get_project(project_id=68a3eec6caa5424ea6bb39d5d6e46c2f)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:48.905089 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d521e72-c26d-4e79-b82d-a3427159a631 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:48.907058 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 190/376] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 24 17:07:48 2022] GET /identity/v3/projects/68a3eec6caa5424ea6bb39d5d6e46c2f => generated 344 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:48.912602 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d521e72-c26d-4e79-b82d-a3427159a631 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:48.942836 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:48.976877 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa1e6db-ba84-4ea5-82d4-b73dbc18b6db None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:49.015480 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ca900e-e1e4-4938-b426-f95e105e346d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:49.016119 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ca900e-e1e4-4938-b426-f95e105e346d None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:49.016399 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ca900e-e1e4-4938-b426-f95e105e346d None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:49.016682 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ca900e-e1e4-4938-b426-f95e105e346d None admin] PATH_INFO: `/v3/groups/3dad4633ec274cfb81918f0a9e56bead` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:49.017861 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ca900e-e1e4-4938-b426-f95e105e346d None admin] RBAC: Authorizing `identity:get_group(group_id=3dad4633ec274cfb81918f0a9e56bead)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:49.019286 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ca900e-e1e4-4938-b426-f95e105e346d None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:49.021164 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 187/377] 10.222.0.18 () {56 vars in 1180 bytes} [Wed Aug 24 17:07:48 2022] GET /identity/v3/groups/3dad4633ec274cfb81918f0a9e56bead => generated 226 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:49.026520 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ca900e-e1e4-4938-b426-f95e105e346d None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:49.058576 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:49.067833 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d521e72-c26d-4e79-b82d-a3427159a631 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:49.103403 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57261747-f46d-4fc2-af73-573d3071a4d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:49.104049 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57261747-f46d-4fc2-af73-573d3071a4d2 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:49.104318 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57261747-f46d-4fc2-af73-573d3071a4d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:49.104615 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57261747-f46d-4fc2-af73-573d3071a4d2 None admin] PATH_INFO: `/v3/groups/0630417565324d3e96df34a8ff800e50` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:49.110132 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57261747-f46d-4fc2-af73-573d3071a4d2 None admin] RBAC: Authorizing `identity:get_group(group_id=0630417565324d3e96df34a8ff800e50)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:49.111610 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57261747-f46d-4fc2-af73-573d3071a4d2 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:49.113239 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 191/378] 10.222.0.18 () {56 vars in 1180 bytes} [Wed Aug 24 17:07:48 2022] GET /identity/v3/groups/0630417565324d3e96df34a8ff800e50 => generated 229 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:07:49.114384 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27724c1-f0a3-40f5-8b0f-4a95e00d89bf None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:49.114663 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27724c1-f0a3-40f5-8b0f-4a95e00d89bf None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:49.116817 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27724c1-f0a3-40f5-8b0f-4a95e00d89bf None None] PATH_INFO: `/` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:49.118358 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 192/379] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:07:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:07:49.125862 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b49901-1f7b-47c7-8aab-8c0e935476b0 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:49.126137 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b49901-1f7b-47c7-8aab-8c0e935476b0 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:49.126411 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b49901-1f7b-47c7-8aab-8c0e935476b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:49.154895 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-39b49901-1f7b-47c7-8aab-8c0e935476b0 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:07:49.156450 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-39b49901-1f7b-47c7-8aab-8c0e935476b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:07:49.184384 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ca900e-e1e4-4938-b426-f95e105e346d None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:49.240833 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b74258e-578d-42a2-b993-1b1ac72cd3c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:49.241577 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b74258e-578d-42a2-b993-1b1ac72cd3c8 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:49.241924 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b74258e-578d-42a2-b993-1b1ac72cd3c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:49.242269 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b74258e-578d-42a2-b993-1b1ac72cd3c8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:49.243100 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b74258e-578d-42a2-b993-1b1ac72cd3c8 None admin] RBAC: Adding query filter params (group.id=3dad4633ec274cfb81918f0a9e56bead, role.id=af19a7a2fae5422da5bd7daea4bbfccf, scope.project.id=94586413811643bb999468e84b659403) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:07:49.243556 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b74258e-578d-42a2-b993-1b1ac72cd3c8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:49.245179 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b74258e-578d-42a2-b993-1b1ac72cd3c8 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:49.253895 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 188/380] 10.222.0.18 () {56 vars in 1376 bytes} [Wed Aug 24 17:07:49 2022] GET /identity/v3/role_assignments?group.id=3dad4633ec274cfb81918f0a9e56bead&scope.project.id=94586413811643bb999468e84b659403&role.id=af19a7a2fae5422da5bd7daea4bbfccf => generated 260 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:49.254618 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b74258e-578d-42a2-b993-1b1ac72cd3c8 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:49.260741 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 193/381] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:07:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:07:49.262930 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:49.376486 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b74258e-578d-42a2-b993-1b1ac72cd3c8 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:49.415953 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714e2e32-5eb9-45a3-8182-436b4f1b98ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:49.416642 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714e2e32-5eb9-45a3-8182-436b4f1b98ae None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:49.416924 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714e2e32-5eb9-45a3-8182-436b4f1b98ae None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:49.417238 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714e2e32-5eb9-45a3-8182-436b4f1b98ae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:49.417981 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714e2e32-5eb9-45a3-8182-436b4f1b98ae None admin] RBAC: Adding query filter params (group.id=0630417565324d3e96df34a8ff800e50, role.id=c5346bc54abf445f9900edf94264c850, scope.project.id=68a3eec6caa5424ea6bb39d5d6e46c2f) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:07:49.418293 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714e2e32-5eb9-45a3-8182-436b4f1b98ae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:49.419821 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714e2e32-5eb9-45a3-8182-436b4f1b98ae None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:49.426362 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 189/382] 10.222.0.18 () {56 vars in 1376 bytes} [Wed Aug 24 17:07:49 2022] GET /identity/v3/role_assignments?group.id=0630417565324d3e96df34a8ff800e50&scope.project.id=68a3eec6caa5424ea6bb39d5d6e46c2f&role.id=c5346bc54abf445f9900edf94264c850 => generated 260 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:49.509100 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3397dacf-70aa-479c-a27d-78868b128ec3 None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:49.509596 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3397dacf-70aa-479c-a27d-78868b128ec3 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:49.509699 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3397dacf-70aa-479c-a27d-78868b128ec3 None None] PATH_INFO: `/` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:49.511283 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 194/383] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:07:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:07:49.517337 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63635da-5539-4a83-ac77-9acfd15d39f4 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:49.517663 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63635da-5539-4a83-ac77-9acfd15d39f4 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:49.517903 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63635da-5539-4a83-ac77-9acfd15d39f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:49.540847 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-a63635da-5539-4a83-ac77-9acfd15d39f4 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:07:49.542081 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-a63635da-5539-4a83-ac77-9acfd15d39f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:07:49.636660 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:49.643152 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 190/384] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:07:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:07:49.705289 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3397dacf-70aa-479c-a27d-78868b128ec3 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:49.793793 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:49.842758 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3397dacf-70aa-479c-a27d-78868b128ec3 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:49.890525 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748a841c-e99d-4828-b3ae-419d365cac7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:49.891117 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748a841c-e99d-4828-b3ae-419d365cac7a None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:49.891393 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748a841c-e99d-4828-b3ae-419d365cac7a None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:49.891661 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748a841c-e99d-4828-b3ae-419d365cac7a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:49.903739 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748a841c-e99d-4828-b3ae-419d365cac7a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:07:49.908467 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-748a841c-e99d-4828-b3ae-419d365cac7a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:07:49.908467 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:07:49.908467 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:07:49.908467 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:07:49.908467 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:07:49.908467 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:07:49.908467 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:07:49.908467 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:07:49.908467 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:07:49.908467 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:07:49.908467 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:07:49.908467 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:07:49.908467 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:07:49.908467 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:07:49.908467 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:07:49.908467 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:07:49.909367 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:07:49.909367 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:07:49.909367 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:07:49.909367 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:07:49.909367 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:07:49.909367 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:07:49.909367 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:07:49.909367 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:07:49.909367 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:07:49.909367 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:07:49.909367 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:07:49.909367 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application with Lock( Aug 24 17:07:49.909367 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:07:49.909367 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:07:49.909367 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:07:49.909367 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:07:49.910157 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:07:49.910157 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:07:49.910157 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:07:49.910157 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:07:49.910157 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:07:49.910157 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:07:49.910157 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:07:49.910157 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:07:49.910157 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:07:49.910157 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:07:49.910157 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:07:49.910157 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:07:49.910157 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 195/385] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 24 17:07:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:07:49.915303 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63635da-5539-4a83-ac77-9acfd15d39f4 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:50.051690 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63635da-5539-4a83-ac77-9acfd15d39f4 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:50.094478 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb876d86-91cc-4541-a740-59f726c70914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:50.095497 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb876d86-91cc-4541-a740-59f726c70914 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:50.095983 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb876d86-91cc-4541-a740-59f726c70914 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:50.096472 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb876d86-91cc-4541-a740-59f726c70914 None admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:50.097429 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb876d86-91cc-4541-a740-59f726c70914 None admin] RBAC: Adding query filter params (name=member) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:07:50.097939 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb876d86-91cc-4541-a740-59f726c70914 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:50.099591 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb876d86-91cc-4541-a740-59f726c70914 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:50.115098 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 191/386] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Aug 24 17:07:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:50.120678 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748a841c-e99d-4828-b3ae-419d365cac7a None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:50.125111 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb876d86-91cc-4541-a740-59f726c70914 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:50.224272 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:50.285828 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748a841c-e99d-4828-b3ae-419d365cac7a None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:50.304748 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb876d86-91cc-4541-a740-59f726c70914 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:50.336321 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:50.347540 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd71b86-6f59-440f-9843-2630279e731c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:50.348178 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd71b86-6f59-440f-9843-2630279e731c None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:50.348453 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd71b86-6f59-440f-9843-2630279e731c None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:50.350825 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd71b86-6f59-440f-9843-2630279e731c None admin] PATH_INFO: `/v3/groups/3dad4633ec274cfb81918f0a9e56bead` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:50.352253 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd71b86-6f59-440f-9843-2630279e731c None admin] RBAC: Authorizing `identity:get_group(group_id=3dad4633ec274cfb81918f0a9e56bead)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:50.353709 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd71b86-6f59-440f-9843-2630279e731c None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:50.359731 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 196/387] 10.222.0.18 () {56 vars in 1180 bytes} [Wed Aug 24 17:07:50 2022] GET /identity/v3/groups/3dad4633ec274cfb81918f0a9e56bead => generated 226 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:50.366119 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd71b86-6f59-440f-9843-2630279e731c None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:50.385654 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3b521a-2cfb-4705-aee0-377f7aeeeff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:50.386315 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3b521a-2cfb-4705-aee0-377f7aeeeff3 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:50.386560 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3b521a-2cfb-4705-aee0-377f7aeeeff3 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:50.386831 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3b521a-2cfb-4705-aee0-377f7aeeeff3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:50.408969 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3b521a-2cfb-4705-aee0-377f7aeeeff3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:07:50.414348 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-9f3b521a-2cfb-4705-aee0-377f7aeeeff3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 17:07:50.414348 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:07:50.414348 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:07:50.414348 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:07:50.414348 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:07:50.414348 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:07:50.414348 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:07:50.414348 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:07:50.414348 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:07:50.414348 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:07:50.414348 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:07:50.414348 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:07:50.414348 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:07:50.414348 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:07:50.414348 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:07:50.414348 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:07:50.415286 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:07:50.415286 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:07:50.415286 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:07:50.415286 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:07:50.415286 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:07:50.415286 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:07:50.415286 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:07:50.415286 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:07:50.415286 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:07:50.415286 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:07:50.415286 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:07:50.415286 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 24 17:07:50.415286 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:07:50.415286 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:07:50.415286 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:07:50.415286 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:07:50.416073 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:07:50.416073 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:07:50.416073 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:07:50.416073 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:07:50.416073 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:07:50.416073 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:07:50.416073 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:07:50.416073 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:07:50.416073 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:07:50.416073 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:07:50.416073 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 17:07:50.416073 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:07:50.416073 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 192/388] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Aug 24 17:07:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:07:50.421419 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3b521a-2cfb-4705-aee0-377f7aeeeff3 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:50.516918 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd71b86-6f59-440f-9843-2630279e731c None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:50.560905 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc67b8eb-51fd-42bc-829d-ee566002aec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:50.561553 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc67b8eb-51fd-42bc-829d-ee566002aec7 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:50.561828 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc67b8eb-51fd-42bc-829d-ee566002aec7 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:50.562100 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc67b8eb-51fd-42bc-829d-ee566002aec7 None admin] PATH_INFO: `/v3/projects/9161a24523d545a8939fb56c3c930122` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:50.563275 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc67b8eb-51fd-42bc-829d-ee566002aec7 None admin] RBAC: Authorizing `identity:get_project(project_id=9161a24523d545a8939fb56c3c930122)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:50.564763 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc67b8eb-51fd-42bc-829d-ee566002aec7 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:50.566378 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 197/389] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 24 17:07:50 2022] GET /identity/v3/projects/9161a24523d545a8939fb56c3c930122 => generated 298 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:07:50.567788 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b60e2f8-15b2-4800-aa46-07c5b8d24882 None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:50.568065 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b60e2f8-15b2-4800-aa46-07c5b8d24882 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:50.568594 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b60e2f8-15b2-4800-aa46-07c5b8d24882 None None] PATH_INFO: `/` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:50.570341 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 198/390] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:07:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:07:50.576290 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac9a7bb-30c3-4af9-ba96-c3d4a355d0d0 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:50.576572 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac9a7bb-30c3-4af9-ba96-c3d4a355d0d0 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:50.576846 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac9a7bb-30c3-4af9-ba96-c3d4a355d0d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:50.604116 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-bac9a7bb-30c3-4af9-ba96-c3d4a355d0d0 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:07:50.605376 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-bac9a7bb-30c3-4af9-ba96-c3d4a355d0d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:07:50.633740 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3b521a-2cfb-4705-aee0-377f7aeeeff3 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:50.662956 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:50.679862 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b5110a-7f33-48f6-a366-37276b499528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:50.680586 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b5110a-7f33-48f6-a366-37276b499528 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:50.680958 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b5110a-7f33-48f6-a366-37276b499528 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:50.681500 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b5110a-7f33-48f6-a366-37276b499528 None admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:50.681951 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b5110a-7f33-48f6-a366-37276b499528 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:07:50.682268 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b5110a-7f33-48f6-a366-37276b499528 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:50.683860 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b5110a-7f33-48f6-a366-37276b499528 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:50.688632 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:50.695439 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 199/391] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:07:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 17:07:50.696208 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 193/392] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 24 17:07:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:07:50.701318 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac9a7bb-30c3-4af9-ba96-c3d4a355d0d0 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:50.703016 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b5110a-7f33-48f6-a366-37276b499528 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:50.830164 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac9a7bb-30c3-4af9-ba96-c3d4a355d0d0 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:50.848175 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b5110a-7f33-48f6-a366-37276b499528 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:50.882995 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638c7bcb-7970-486b-80e8-58e1cf97b880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:50.883705 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638c7bcb-7970-486b-80e8-58e1cf97b880 None admin] REQUEST_METHOD: `PUT` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:50.883957 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638c7bcb-7970-486b-80e8-58e1cf97b880 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:50.884231 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638c7bcb-7970-486b-80e8-58e1cf97b880 None admin] PATH_INFO: `/v3/projects/9161a24523d545a8939fb56c3c930122/groups/3dad4633ec274cfb81918f0a9e56bead/roles/60dc7e16aee746d5a3e8aa27d7d8525c` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:50.890218 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638c7bcb-7970-486b-80e8-58e1cf97b880 None admin] RBAC: Authorizing `identity:create_grant(project_id=9161a24523d545a8939fb56c3c930122, group_id=3dad4633ec274cfb81918f0a9e56bead, role_id=60dc7e16aee746d5a3e8aa27d7d8525c)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:50.890218 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638c7bcb-7970-486b-80e8-58e1cf97b880 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:50.894339 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac82e38-05ba-411d-9ec3-c8f7d5e45671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:50.895280 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac82e38-05ba-411d-9ec3-c8f7d5e45671 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:50.895813 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac82e38-05ba-411d-9ec3-c8f7d5e45671 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:50.896308 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac82e38-05ba-411d-9ec3-c8f7d5e45671 None admin] PATH_INFO: `/v3/groups/3dad4633ec274cfb81918f0a9e56bead` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:50.897756 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac82e38-05ba-411d-9ec3-c8f7d5e45671 None admin] RBAC: Authorizing `identity:get_group(group_id=3dad4633ec274cfb81918f0a9e56bead)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:50.899532 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac82e38-05ba-411d-9ec3-c8f7d5e45671 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:50.901875 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 194/393] 10.222.0.18 () {56 vars in 1180 bytes} [Wed Aug 24 17:07:50 2022] GET /identity/v3/groups/3dad4633ec274cfb81918f0a9e56bead => generated 226 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:07:50.907241 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac82e38-05ba-411d-9ec3-c8f7d5e45671 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:50.964337 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:50.967790 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 200/394] 10.222.0.18 () {58 vars in 1442 bytes} [Wed Aug 24 17:07:50 2022] PUT /identity/v3/projects/9161a24523d545a8939fb56c3c930122/groups/3dad4633ec274cfb81918f0a9e56bead/roles/60dc7e16aee746d5a3e8aa27d7d8525c => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:07:51.023026 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac82e38-05ba-411d-9ec3-c8f7d5e45671 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:51.062929 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a22d8f-2859-42e9-8380-cc90e0b69265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:51.063583 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a22d8f-2859-42e9-8380-cc90e0b69265 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:51.063867 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a22d8f-2859-42e9-8380-cc90e0b69265 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:51.064143 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a22d8f-2859-42e9-8380-cc90e0b69265 None admin] PATH_INFO: `/v3/projects/9161a24523d545a8939fb56c3c930122` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:51.065361 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a22d8f-2859-42e9-8380-cc90e0b69265 None admin] RBAC: Authorizing `identity:get_project(project_id=9161a24523d545a8939fb56c3c930122)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:51.066848 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a22d8f-2859-42e9-8380-cc90e0b69265 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:51.068953 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 195/395] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 24 17:07:50 2022] GET /identity/v3/projects/9161a24523d545a8939fb56c3c930122 => generated 298 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:51.074963 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638c7bcb-7970-486b-80e8-58e1cf97b880 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:51.108158 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a22d8f-2859-42e9-8380-cc90e0b69265 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:51.173843 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:51.246462 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638c7bcb-7970-486b-80e8-58e1cf97b880 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:51.275116 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a22d8f-2859-42e9-8380-cc90e0b69265 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:51.297006 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21749ad9-a023-40dc-ba3e-9112db78da4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:51.298342 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21749ad9-a023-40dc-ba3e-9112db78da4f None admin] REQUEST_METHOD: `PUT` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:51.298829 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21749ad9-a023-40dc-ba3e-9112db78da4f None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:51.299534 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21749ad9-a023-40dc-ba3e-9112db78da4f None admin] PATH_INFO: `/v3/projects/9161a24523d545a8939fb56c3c930122/groups/3dad4633ec274cfb81918f0a9e56bead/roles/af19a7a2fae5422da5bd7daea4bbfccf` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:51.300231 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:51.302366 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21749ad9-a023-40dc-ba3e-9112db78da4f None admin] RBAC: Authorizing `identity:create_grant(project_id=9161a24523d545a8939fb56c3c930122, group_id=3dad4633ec274cfb81918f0a9e56bead, role_id=af19a7a2fae5422da5bd7daea4bbfccf)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:51.305958 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21749ad9-a023-40dc-ba3e-9112db78da4f None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:51.321784 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6465f536-7b2a-4c77-b0f6-c35db27f6f41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:51.322415 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6465f536-7b2a-4c77-b0f6-c35db27f6f41 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:51.322693 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6465f536-7b2a-4c77-b0f6-c35db27f6f41 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:51.329260 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6465f536-7b2a-4c77-b0f6-c35db27f6f41 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:51.336342 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6465f536-7b2a-4c77-b0f6-c35db27f6f41 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:07:51.341815 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-6465f536-7b2a-4c77-b0f6-c35db27f6f41 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:07:51.341815 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:07:51.341815 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:07:51.341815 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:07:51.341815 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:07:51.341815 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:07:51.341815 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:07:51.341815 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:07:51.341815 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:07:51.341815 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:07:51.341815 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:07:51.341815 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:07:51.341815 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:07:51.341815 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:07:51.341815 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:07:51.341815 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:07:51.342699 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:07:51.342699 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:07:51.342699 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:07:51.342699 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:07:51.342699 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:07:51.342699 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:07:51.342699 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:07:51.342699 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:07:51.342699 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:07:51.342699 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:07:51.342699 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:07:51.342699 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 24 17:07:51.342699 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:07:51.342699 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:07:51.342699 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:07:51.342699 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:07:51.343527 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:07:51.343527 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:07:51.343527 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:07:51.343527 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:07:51.343527 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:07:51.343527 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:07:51.343527 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:07:51.343527 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:07:51.343527 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:07:51.343527 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:07:51.343527 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:07:51.343527 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:07:51.346975 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 196/396] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 24 17:07:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:07:51.349072 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6465f536-7b2a-4c77-b0f6-c35db27f6f41 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:51.384550 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 201/397] 10.222.0.18 () {58 vars in 1442 bytes} [Wed Aug 24 17:07:51 2022] PUT /identity/v3/projects/9161a24523d545a8939fb56c3c930122/groups/3dad4633ec274cfb81918f0a9e56bead/roles/af19a7a2fae5422da5bd7daea4bbfccf => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:07:51.475055 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6465f536-7b2a-4c77-b0f6-c35db27f6f41 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:51.524405 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93992d60-11d2-46c6-b21e-e603559edecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:51.525050 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93992d60-11d2-46c6-b21e-e603559edecc None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:51.525324 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93992d60-11d2-46c6-b21e-e603559edecc None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:51.525601 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93992d60-11d2-46c6-b21e-e603559edecc None admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:51.526296 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93992d60-11d2-46c6-b21e-e603559edecc None admin] RBAC: Adding query filter params (name=member) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:07:51.526587 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93992d60-11d2-46c6-b21e-e603559edecc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:51.527995 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93992d60-11d2-46c6-b21e-e603559edecc None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:51.542718 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 197/398] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Aug 24 17:07:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:07:51.548517 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21749ad9-a023-40dc-ba3e-9112db78da4f None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:51.696454 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21749ad9-a023-40dc-ba3e-9112db78da4f None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:51.716188 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:51.734018 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ba0679-a46e-414b-9c25-20f213623071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:51.734807 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ba0679-a46e-414b-9c25-20f213623071 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:51.735221 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ba0679-a46e-414b-9c25-20f213623071 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:51.735592 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ba0679-a46e-414b-9c25-20f213623071 None admin] PATH_INFO: `/v3/groups/3dad4633ec274cfb81918f0a9e56bead` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:51.736826 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ba0679-a46e-414b-9c25-20f213623071 None admin] RBAC: Authorizing `identity:get_group(group_id=3dad4633ec274cfb81918f0a9e56bead)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:51.738372 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ba0679-a46e-414b-9c25-20f213623071 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:51.740390 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 202/399] 10.222.0.18 () {56 vars in 1180 bytes} [Wed Aug 24 17:07:51 2022] GET /identity/v3/groups/3dad4633ec274cfb81918f0a9e56bead => generated 226 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:51.745914 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93992d60-11d2-46c6-b21e-e603559edecc None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:51.792980 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:51.855183 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93992d60-11d2-46c6-b21e-e603559edecc None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:51.893064 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972460a3-85de-4f62-984a-551240733e25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:51.893718 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972460a3-85de-4f62-984a-551240733e25 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:51.894000 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972460a3-85de-4f62-984a-551240733e25 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:51.894278 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972460a3-85de-4f62-984a-551240733e25 None admin] PATH_INFO: `/v3/projects/94586413811643bb999468e84b659403` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:51.895479 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972460a3-85de-4f62-984a-551240733e25 None admin] RBAC: Authorizing `identity:get_project(project_id=94586413811643bb999468e84b659403)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:51.896942 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972460a3-85de-4f62-984a-551240733e25 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:51.898982 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 198/400] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 24 17:07:51 2022] GET /identity/v3/projects/94586413811643bb999468e84b659403 => generated 302 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:07:51.907839 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ba0679-a46e-414b-9c25-20f213623071 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:51.990494 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fd6aac-1f91-4bb8-836e-cceebb60f0cc None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:51.990790 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fd6aac-1f91-4bb8-836e-cceebb60f0cc None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:51.991005 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fd6aac-1f91-4bb8-836e-cceebb60f0cc None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:51.992362 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 199/401] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:07:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:07:52.000956 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb14888b-01bf-4b31-a022-36f2e43828fd None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:52.001265 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb14888b-01bf-4b31-a022-36f2e43828fd None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:52.001520 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb14888b-01bf-4b31-a022-36f2e43828fd None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:52.002829 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 200/402] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:07:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 17:07:52.009480 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9a2920-0e36-4e6b-b285-708e5b470aa2 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:52.009769 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9a2920-0e36-4e6b-b285-708e5b470aa2 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:52.010046 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9a2920-0e36-4e6b-b285-708e5b470aa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:52.029844 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-8e9a2920-0e36-4e6b-b285-708e5b470aa2 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:07:52.031089 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-8e9a2920-0e36-4e6b-b285-708e5b470aa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:07:52.057250 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ba0679-a46e-414b-9c25-20f213623071 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:52.104714 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc1600f-ad1f-4822-b088-6906e9846efb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:52.105760 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc1600f-ad1f-4822-b088-6906e9846efb None admin] REQUEST_METHOD: `PUT` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:52.106112 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc1600f-ad1f-4822-b088-6906e9846efb None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:52.106457 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc1600f-ad1f-4822-b088-6906e9846efb None admin] PATH_INFO: `/v3/projects/94586413811643bb999468e84b659403/groups/3dad4633ec274cfb81918f0a9e56bead/roles/60dc7e16aee746d5a3e8aa27d7d8525c` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:52.109928 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc1600f-ad1f-4822-b088-6906e9846efb None admin] RBAC: Authorizing `identity:create_grant(project_id=94586413811643bb999468e84b659403, group_id=3dad4633ec274cfb81918f0a9e56bead, role_id=60dc7e16aee746d5a3e8aa27d7d8525c)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:52.111010 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc1600f-ad1f-4822-b088-6906e9846efb None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:52.122580 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 201/403] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:07:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 17:07:52.124367 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed351a8-5034-466a-af2c-8b40bbb77eeb None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:52.124854 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed351a8-5034-466a-af2c-8b40bbb77eeb None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:52.125285 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed351a8-5034-466a-af2c-8b40bbb77eeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:52.148192 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-0ed351a8-5034-466a-af2c-8b40bbb77eeb None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:07:52.149714 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-0ed351a8-5034-466a-af2c-8b40bbb77eeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:07:52.187780 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:52.198391 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 203/404] 10.222.0.18 () {58 vars in 1442 bytes} [Wed Aug 24 17:07:51 2022] PUT /identity/v3/projects/94586413811643bb999468e84b659403/groups/3dad4633ec274cfb81918f0a9e56bead/roles/60dc7e16aee746d5a3e8aa27d7d8525c => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:07:52.242142 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 202/405] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:07:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:07:52.438570 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc1600f-ad1f-4822-b088-6906e9846efb None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:52.485343 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:52.541340 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc1600f-ad1f-4822-b088-6906e9846efb None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:52.578546 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b33432-6de2-48be-8f14-f323736c4d9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:52.579301 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b33432-6de2-48be-8f14-f323736c4d9c None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:52.579486 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b33432-6de2-48be-8f14-f323736c4d9c None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:52.579757 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b33432-6de2-48be-8f14-f323736c4d9c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:52.590824 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b33432-6de2-48be-8f14-f323736c4d9c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:07:52.595644 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-e4b33432-6de2-48be-8f14-f323736c4d9c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 17:07:52.595644 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:07:52.595644 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:07:52.595644 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:07:52.595644 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:07:52.595644 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:07:52.595644 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:07:52.595644 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:07:52.595644 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:07:52.595644 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:07:52.595644 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:07:52.595644 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:07:52.595644 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:07:52.595644 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:07:52.595644 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:07:52.595644 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:07:52.596539 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:07:52.596539 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:07:52.596539 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:07:52.596539 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:07:52.596539 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:07:52.596539 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:07:52.596539 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:07:52.596539 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:07:52.596539 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:07:52.596539 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:07:52.596539 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:07:52.596539 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application with Lock( Aug 24 17:07:52.596539 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:07:52.596539 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:07:52.596539 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:07:52.596539 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:07:52.597723 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:07:52.597723 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:07:52.597723 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:07:52.597723 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:07:52.597723 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:07:52.597723 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:07:52.597723 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:07:52.597723 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:07:52.597723 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:07:52.597723 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:07:52.597723 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 17:07:52.597723 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:07:52.597723 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 204/406] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Aug 24 17:07:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:07:52.602707 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed351a8-5034-466a-af2c-8b40bbb77eeb None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:52.664827 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b33432-6de2-48be-8f14-f323736c4d9c None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:52.713217 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed351a8-5034-466a-af2c-8b40bbb77eeb None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:52.748743 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ed7a66-ecc9-4799-8568-f1dbbd488450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:52.749357 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ed7a66-ecc9-4799-8568-f1dbbd488450 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:52.749636 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ed7a66-ecc9-4799-8568-f1dbbd488450 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:52.749908 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ed7a66-ecc9-4799-8568-f1dbbd488450 None admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:52.750828 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ed7a66-ecc9-4799-8568-f1dbbd488450 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:07:52.751154 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ed7a66-ecc9-4799-8568-f1dbbd488450 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:52.752525 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ed7a66-ecc9-4799-8568-f1dbbd488450 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:52.761099 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 203/407] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 24 17:07:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:52.766597 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ed7a66-ecc9-4799-8568-f1dbbd488450 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:52.812003 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:52.879278 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ed7a66-ecc9-4799-8568-f1dbbd488450 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:52.912926 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b33432-6de2-48be-8f14-f323736c4d9c None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:52.917467 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a4493b-d9c2-4254-96d1-a458c7698031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:52.918127 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a4493b-d9c2-4254-96d1-a458c7698031 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:52.918410 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a4493b-d9c2-4254-96d1-a458c7698031 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:52.918687 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a4493b-d9c2-4254-96d1-a458c7698031 None admin] PATH_INFO: `/v3/groups/3dad4633ec274cfb81918f0a9e56bead` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:52.920369 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a4493b-d9c2-4254-96d1-a458c7698031 None admin] RBAC: Authorizing `identity:get_group(group_id=3dad4633ec274cfb81918f0a9e56bead)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:52.921832 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a4493b-d9c2-4254-96d1-a458c7698031 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:52.923508 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 204/408] 10.222.0.18 () {56 vars in 1180 bytes} [Wed Aug 24 17:07:52 2022] GET /identity/v3/groups/3dad4633ec274cfb81918f0a9e56bead => generated 226 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:07:52.928712 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a4493b-d9c2-4254-96d1-a458c7698031 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:52.943284 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:53.035299 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:53.036143 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fb463a-cad9-4d22-b42f-4ae19f6a59f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:53.036800 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fb463a-cad9-4d22-b42f-4ae19f6a59f8 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:53.037080 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fb463a-cad9-4d22-b42f-4ae19f6a59f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:53.037352 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fb463a-cad9-4d22-b42f-4ae19f6a59f8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:53.040596 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a4493b-d9c2-4254-96d1-a458c7698031 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:53.067888 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fb463a-cad9-4d22-b42f-4ae19f6a59f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:07:53.073159 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-33fb463a-cad9-4d22-b42f-4ae19f6a59f8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:07:53.073159 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:07:53.073159 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:07:53.073159 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:07:53.073159 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:07:53.073159 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:07:53.073159 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:07:53.073159 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:07:53.073159 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:07:53.073159 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:07:53.073159 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:07:53.073159 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:07:53.073159 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:07:53.073159 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:07:53.073159 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:07:53.073159 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:07:53.074018 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:07:53.074018 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:07:53.074018 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:07:53.074018 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:07:53.074018 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:07:53.074018 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:07:53.074018 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:07:53.074018 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:07:53.074018 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:07:53.074018 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:07:53.074018 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:07:53.074018 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application with Lock( Aug 24 17:07:53.074018 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:07:53.074018 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:07:53.074018 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:07:53.074018 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:07:53.074957 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:07:53.074957 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:07:53.074957 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:07:53.074957 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:07:53.074957 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:07:53.074957 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:07:53.074957 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:07:53.074957 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:07:53.074957 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:07:53.074957 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:07:53.074957 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:07:53.074957 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:07:53.080678 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 205/409] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Aug 24 17:07:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 413 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:07:53.082156 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fb463a-cad9-4d22-b42f-4ae19f6a59f8 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:53.105539 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664b3ba5-b91f-4f67-b096-4e7a0e8069ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:53.106150 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664b3ba5-b91f-4f67-b096-4e7a0e8069ca None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:53.106429 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664b3ba5-b91f-4f67-b096-4e7a0e8069ca None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:53.106758 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664b3ba5-b91f-4f67-b096-4e7a0e8069ca None admin] PATH_INFO: `/v3/projects/94586413811643bb999468e84b659403` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:53.109116 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664b3ba5-b91f-4f67-b096-4e7a0e8069ca None admin] RBAC: Authorizing `identity:get_project(project_id=94586413811643bb999468e84b659403)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:53.110540 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664b3ba5-b91f-4f67-b096-4e7a0e8069ca None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:53.112686 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 205/410] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 24 17:07:52 2022] GET /identity/v3/projects/94586413811643bb999468e84b659403 => generated 302 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:53.121360 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664b3ba5-b91f-4f67-b096-4e7a0e8069ca None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:53.281935 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664b3ba5-b91f-4f67-b096-4e7a0e8069ca None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:53.333548 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1f140e-5ec1-461a-a34f-9e1032307997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:53.334930 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1f140e-5ec1-461a-a34f-9e1032307997 None admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:53.334930 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1f140e-5ec1-461a-a34f-9e1032307997 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:53.334930 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1f140e-5ec1-461a-a34f-9e1032307997 None admin] PATH_INFO: `/v3/projects/94586413811643bb999468e84b659403/groups/3dad4633ec274cfb81918f0a9e56bead/roles/af19a7a2fae5422da5bd7daea4bbfccf` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:53.338929 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1f140e-5ec1-461a-a34f-9e1032307997 None admin] RBAC: Authorizing `identity:create_grant(project_id=94586413811643bb999468e84b659403, group_id=3dad4633ec274cfb81918f0a9e56bead, role_id=af19a7a2fae5422da5bd7daea4bbfccf)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:53.338929 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1f140e-5ec1-461a-a34f-9e1032307997 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:53.354123 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fb463a-cad9-4d22-b42f-4ae19f6a59f8 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:53.408858 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edbd7ab-bd8f-4b92-a8d0-25a93ee5b902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:53.411008 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edbd7ab-bd8f-4b92-a8d0-25a93ee5b902 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:53.411008 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edbd7ab-bd8f-4b92-a8d0-25a93ee5b902 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:53.411008 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edbd7ab-bd8f-4b92-a8d0-25a93ee5b902 None admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:53.411008 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edbd7ab-bd8f-4b92-a8d0-25a93ee5b902 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:07:53.412673 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edbd7ab-bd8f-4b92-a8d0-25a93ee5b902 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:53.414069 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edbd7ab-bd8f-4b92-a8d0-25a93ee5b902 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:53.428142 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 206/411] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 24 17:07:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:07:53.437620 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edbd7ab-bd8f-4b92-a8d0-25a93ee5b902 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:53.471478 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:53.487539 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 206/412] 10.222.0.18 () {58 vars in 1442 bytes} [Wed Aug 24 17:07:53 2022] PUT /identity/v3/projects/94586413811643bb999468e84b659403/groups/3dad4633ec274cfb81918f0a9e56bead/roles/af19a7a2fae5422da5bd7daea4bbfccf => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:07:53.625305 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edbd7ab-bd8f-4b92-a8d0-25a93ee5b902 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:53.704605 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55d99af-3a98-4d42-b28f-27bc41c94728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:53.705273 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55d99af-3a98-4d42-b28f-27bc41c94728 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:53.705548 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55d99af-3a98-4d42-b28f-27bc41c94728 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:53.705821 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55d99af-3a98-4d42-b28f-27bc41c94728 None admin] PATH_INFO: `/v3/groups/0630417565324d3e96df34a8ff800e50` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:53.711762 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55d99af-3a98-4d42-b28f-27bc41c94728 None admin] RBAC: Authorizing `identity:get_group(group_id=0630417565324d3e96df34a8ff800e50)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:53.713245 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55d99af-3a98-4d42-b28f-27bc41c94728 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:53.715223 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 207/413] 10.222.0.18 () {56 vars in 1180 bytes} [Wed Aug 24 17:07:53 2022] GET /identity/v3/groups/0630417565324d3e96df34a8ff800e50 => generated 229 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:53.724057 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1f140e-5ec1-461a-a34f-9e1032307997 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:53.757994 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:53.817342 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd6c00c-3043-45dd-9a61-d74507189fdd None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:53.817669 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd6c00c-3043-45dd-9a61-d74507189fdd None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:53.817997 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd6c00c-3043-45dd-9a61-d74507189fdd None None] PATH_INFO: `/` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:53.820445 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 208/414] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:07:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:07:53.824847 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3add5304-12c9-4888-adb1-7415c251e6a7 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:53.825132 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3add5304-12c9-4888-adb1-7415c251e6a7 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:53.825437 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3add5304-12c9-4888-adb1-7415c251e6a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:53.848137 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1f140e-5ec1-461a-a34f-9e1032307997 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:53.860088 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-3add5304-12c9-4888-adb1-7415c251e6a7 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:07:53.861143 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-3add5304-12c9-4888-adb1-7415c251e6a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:07:53.886602 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b3bfac-57e7-4e2c-822c-1b8169822ad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:53.887587 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b3bfac-57e7-4e2c-822c-1b8169822ad8 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:53.888159 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b3bfac-57e7-4e2c-822c-1b8169822ad8 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:53.888808 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b3bfac-57e7-4e2c-822c-1b8169822ad8 None admin] PATH_INFO: `/v3/projects/68a3eec6caa5424ea6bb39d5d6e46c2f` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:53.890444 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b3bfac-57e7-4e2c-822c-1b8169822ad8 None admin] RBAC: Authorizing `identity:get_project(project_id=68a3eec6caa5424ea6bb39d5d6e46c2f)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:53.892280 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b3bfac-57e7-4e2c-822c-1b8169822ad8 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:53.894143 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 207/415] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 24 17:07:53 2022] GET /identity/v3/projects/68a3eec6caa5424ea6bb39d5d6e46c2f => generated 344 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:53.901635 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b3bfac-57e7-4e2c-822c-1b8169822ad8 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:53.991855 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 209/416] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:07:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 17:07:53.993419 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4851fb6-5e9d-4cb8-9f54-0197b846d5e6 None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:53.993706 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4851fb6-5e9d-4cb8-9f54-0197b846d5e6 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:53.993977 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4851fb6-5e9d-4cb8-9f54-0197b846d5e6 None None] PATH_INFO: `/` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:53.995769 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 210/417] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:07:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:07:54.001369 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012286ec-abee-4a5b-94cc-1cc18e5b5552 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:54.001648 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012286ec-abee-4a5b-94cc-1cc18e5b5552 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:54.001914 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012286ec-abee-4a5b-94cc-1cc18e5b5552 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:54.013126 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:54.019860 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-012286ec-abee-4a5b-94cc-1cc18e5b5552 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:07:54.021041 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-012286ec-abee-4a5b-94cc-1cc18e5b5552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:07:54.021433 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b3bfac-57e7-4e2c-822c-1b8169822ad8 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:54.042269 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:54.073522 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdf8e26-3624-410c-9dbe-c24cf8c34625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:54.074190 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdf8e26-3624-410c-9dbe-c24cf8c34625 None admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:54.074472 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdf8e26-3624-410c-9dbe-c24cf8c34625 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:54.074744 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdf8e26-3624-410c-9dbe-c24cf8c34625 None admin] PATH_INFO: `/v3/projects/68a3eec6caa5424ea6bb39d5d6e46c2f/groups/0630417565324d3e96df34a8ff800e50/roles/c5346bc54abf445f9900edf94264c850` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:54.076721 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdf8e26-3624-410c-9dbe-c24cf8c34625 None admin] RBAC: Authorizing `identity:create_grant(project_id=68a3eec6caa5424ea6bb39d5d6e46c2f, group_id=0630417565324d3e96df34a8ff800e50, role_id=c5346bc54abf445f9900edf94264c850)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:54.078162 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdf8e26-3624-410c-9dbe-c24cf8c34625 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:54.090025 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 211/418] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:07:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:07:54.195842 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 208/419] 10.222.0.18 () {58 vars in 1442 bytes} [Wed Aug 24 17:07:53 2022] PUT /identity/v3/projects/68a3eec6caa5424ea6bb39d5d6e46c2f/groups/0630417565324d3e96df34a8ff800e50/roles/c5346bc54abf445f9900edf94264c850 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:07:54.354657 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012286ec-abee-4a5b-94cc-1cc18e5b5552 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:54.427715 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdf8e26-3624-410c-9dbe-c24cf8c34625 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:54.463877 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012286ec-abee-4a5b-94cc-1cc18e5b5552 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:54.523083 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0086a709-299a-42cb-9eea-1e5c4d3c3cb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:54.523841 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0086a709-299a-42cb-9eea-1e5c4d3c3cb0 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:54.524115 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0086a709-299a-42cb-9eea-1e5c4d3c3cb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:54.524380 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0086a709-299a-42cb-9eea-1e5c4d3c3cb0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:54.549192 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdf8e26-3624-410c-9dbe-c24cf8c34625 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:54.560597 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0086a709-299a-42cb-9eea-1e5c4d3c3cb0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:07:54.565726 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-0086a709-299a-42cb-9eea-1e5c4d3c3cb0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:07:54.565726 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:07:54.565726 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:07:54.565726 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:07:54.565726 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:07:54.565726 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:07:54.565726 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:07:54.565726 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:07:54.565726 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:07:54.565726 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:07:54.565726 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:07:54.565726 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:07:54.565726 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:07:54.565726 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:07:54.565726 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:07:54.565726 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:07:54.566597 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:07:54.566597 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:07:54.566597 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:07:54.566597 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:07:54.566597 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:07:54.566597 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:07:54.566597 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:07:54.566597 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:07:54.566597 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:07:54.566597 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:07:54.566597 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:07:54.566597 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application with Lock( Aug 24 17:07:54.566597 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:07:54.566597 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:07:54.566597 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:07:54.566597 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:07:54.567434 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:07:54.567434 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:07:54.567434 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:07:54.567434 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:07:54.567434 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:07:54.567434 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:07:54.567434 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:07:54.567434 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:07:54.567434 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:07:54.567434 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:07:54.567434 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:07:54.567434 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:07:54.567434 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 212/420] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 24 17:07:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:07:54.572644 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0086a709-299a-42cb-9eea-1e5c4d3c3cb0 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:54.598087 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:54.618790 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56aa6e0e-857d-4c05-8c89-4ca9330c8032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:54.619498 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56aa6e0e-857d-4c05-8c89-4ca9330c8032 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:54.619776 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56aa6e0e-857d-4c05-8c89-4ca9330c8032 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:54.620047 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56aa6e0e-857d-4c05-8c89-4ca9330c8032 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:54.641850 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56aa6e0e-857d-4c05-8c89-4ca9330c8032 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:07:54.648686 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-56aa6e0e-857d-4c05-8c89-4ca9330c8032 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 17:07:54.648686 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:07:54.648686 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:07:54.648686 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:07:54.648686 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:07:54.648686 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:07:54.648686 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:07:54.648686 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:07:54.648686 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:07:54.648686 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:07:54.648686 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:07:54.648686 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:07:54.648686 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:07:54.648686 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:07:54.648686 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:07:54.648686 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:07:54.649564 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:07:54.649564 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:07:54.649564 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:07:54.649564 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:07:54.649564 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:07:54.649564 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:07:54.649564 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:07:54.649564 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:07:54.649564 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:07:54.649564 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:07:54.649564 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:07:54.649564 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 24 17:07:54.649564 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:07:54.649564 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:07:54.649564 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:07:54.649564 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:07:54.650360 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:07:54.650360 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:07:54.650360 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:07:54.650360 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:07:54.650360 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:07:54.650360 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:07:54.650360 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:07:54.650360 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:07:54.650360 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:07:54.650360 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:07:54.650360 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 17:07:54.650360 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:07:54.651449 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 209/421] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Aug 24 17:07:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:07:54.658726 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56aa6e0e-857d-4c05-8c89-4ca9330c8032 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:54.688380 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:54.751704 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0086a709-299a-42cb-9eea-1e5c4d3c3cb0 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:54.798407 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56aa6e0e-857d-4c05-8c89-4ca9330c8032 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:54.838578 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b955cea0-3cae-4c3a-b04b-4d38675d71b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:54.840145 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b955cea0-3cae-4c3a-b04b-4d38675d71b5 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:54.841689 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b955cea0-3cae-4c3a-b04b-4d38675d71b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:54.842600 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b955cea0-3cae-4c3a-b04b-4d38675d71b5 None admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:54.843893 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b955cea0-3cae-4c3a-b04b-4d38675d71b5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:07:54.844984 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b955cea0-3cae-4c3a-b04b-4d38675d71b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:54.845801 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac5d547-c8f1-4c21-ae16-9196105fd911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:54.846649 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac5d547-c8f1-4c21-ae16-9196105fd911 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:54.847169 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac5d547-c8f1-4c21-ae16-9196105fd911 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:54.847652 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac5d547-c8f1-4c21-ae16-9196105fd911 None admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:54.848528 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac5d547-c8f1-4c21-ae16-9196105fd911 None admin] RBAC: Adding query filter params (name=member) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:07:54.848633 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b955cea0-3cae-4c3a-b04b-4d38675d71b5 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:54.849122 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac5d547-c8f1-4c21-ae16-9196105fd911 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:54.850689 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac5d547-c8f1-4c21-ae16-9196105fd911 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:54.858721 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 210/422] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 24 17:07:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:54.864586 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b955cea0-3cae-4c3a-b04b-4d38675d71b5 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:54.869732 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 213/423] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Aug 24 17:07:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:07:54.876545 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0a81d1-d849-4d3c-b4bd-bd22aadc552c None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:54.876909 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0a81d1-d849-4d3c-b4bd-bd22aadc552c None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:54.877255 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0a81d1-d849-4d3c-b4bd-bd22aadc552c None None] PATH_INFO: `/` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:54.878730 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 214/424] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:07:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 17:07:54.880828 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0a81d1-d849-4d3c-b4bd-bd22aadc552c None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:54.951728 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:54.975603 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b955cea0-3cae-4c3a-b04b-4d38675d71b5 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:54.983313 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:54.993011 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0a81d1-d849-4d3c-b4bd-bd22aadc552c None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:55.011387 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4020a6fc-7acd-48a6-a913-6ecece6f2fdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:55.011926 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4020a6fc-7acd-48a6-a913-6ecece6f2fdf None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:55.012202 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4020a6fc-7acd-48a6-a913-6ecece6f2fdf None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:55.012470 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4020a6fc-7acd-48a6-a913-6ecece6f2fdf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:55.031759 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5c3f0b-21bd-42b0-8edd-a46bb94f492e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:55.032657 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5c3f0b-21bd-42b0-8edd-a46bb94f492e None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:55.033208 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5c3f0b-21bd-42b0-8edd-a46bb94f492e None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:55.033656 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5c3f0b-21bd-42b0-8edd-a46bb94f492e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:55.086678 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4020a6fc-7acd-48a6-a913-6ecece6f2fdf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:55.088274 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4020a6fc-7acd-48a6-a913-6ecece6f2fdf None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:55.105942 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5c3f0b-21bd-42b0-8edd-a46bb94f492e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:55.107815 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5c3f0b-21bd-42b0-8edd-a46bb94f492e None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:55.169966 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:55.189368 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:55.201679 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 211/425] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 24 17:07:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 17:07:55.203329 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c45670-7721-4092-bd72-585b1ee7e395 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:55.203778 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c45670-7721-4092-bd72-585b1ee7e395 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:55.204246 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c45670-7721-4092-bd72-585b1ee7e395 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:55.218778 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 215/426] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 24 17:07:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 17:07:55.218778 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5c3f0b-21bd-42b0-8edd-a46bb94f492e None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:55.228902 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-67c45670-7721-4092-bd72-585b1ee7e395 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:07:55.230529 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-67c45670-7721-4092-bd72-585b1ee7e395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:07:55.308260 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 212/427] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:07:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:07:55.309351 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c45670-7721-4092-bd72-585b1ee7e395 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:55.360567 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5c3f0b-21bd-42b0-8edd-a46bb94f492e None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:55.425836 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86582f3-a676-4e8e-876a-8b951b83f6b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:55.426881 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86582f3-a676-4e8e-876a-8b951b83f6b7 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:55.428846 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c45670-7721-4092-bd72-585b1ee7e395 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:55.429306 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86582f3-a676-4e8e-876a-8b951b83f6b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:55.429749 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86582f3-a676-4e8e-876a-8b951b83f6b7 None admin] PATH_INFO: `/v3/projects/9161a24523d545a8939fb56c3c930122` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:55.432603 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86582f3-a676-4e8e-876a-8b951b83f6b7 None admin] RBAC: Authorizing `identity:get_project(project_id=9161a24523d545a8939fb56c3c930122)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:55.436434 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86582f3-a676-4e8e-876a-8b951b83f6b7 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:55.443270 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 216/428] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 24 17:07:55 2022] GET /identity/v3/projects/9161a24523d545a8939fb56c3c930122 => generated 298 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:55.451386 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:55.452643 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86582f3-a676-4e8e-876a-8b951b83f6b7 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:55.481284 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad549b7-7e41-458e-b812-f5151ff8ec00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:55.482538 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad549b7-7e41-458e-b812-f5151ff8ec00 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:55.482816 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad549b7-7e41-458e-b812-f5151ff8ec00 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:55.483113 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad549b7-7e41-458e-b812-f5151ff8ec00 None admin] PATH_INFO: `/v3/projects/9161a24523d545a8939fb56c3c930122` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:55.484485 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad549b7-7e41-458e-b812-f5151ff8ec00 None admin] RBAC: Authorizing `identity:get_project(project_id=9161a24523d545a8939fb56c3c930122)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:55.486094 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad549b7-7e41-458e-b812-f5151ff8ec00 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:55.488077 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 213/429] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 24 17:07:55 2022] GET /identity/v3/projects/9161a24523d545a8939fb56c3c930122 => generated 298 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:55.493460 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad549b7-7e41-458e-b812-f5151ff8ec00 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:55.581637 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:55.620808 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad549b7-7e41-458e-b812-f5151ff8ec00 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:55.657401 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f184bb-c21a-4746-b844-3aab3a47f396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:55.658357 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f184bb-c21a-4746-b844-3aab3a47f396 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:55.658808 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f184bb-c21a-4746-b844-3aab3a47f396 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:55.659290 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f184bb-c21a-4746-b844-3aab3a47f396 None admin] PATH_INFO: `/v3/groups/3dad4633ec274cfb81918f0a9e56bead` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:55.660584 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f184bb-c21a-4746-b844-3aab3a47f396 None admin] RBAC: Authorizing `identity:get_group(group_id=3dad4633ec274cfb81918f0a9e56bead)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:55.662476 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f184bb-c21a-4746-b844-3aab3a47f396 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:55.664296 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 214/430] 10.222.0.18 () {56 vars in 1180 bytes} [Wed Aug 24 17:07:55 2022] GET /identity/v3/groups/3dad4633ec274cfb81918f0a9e56bead => generated 226 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:07:55.670612 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f184bb-c21a-4746-b844-3aab3a47f396 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:55.705716 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86582f3-a676-4e8e-876a-8b951b83f6b7 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:55.717224 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:55.742401 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ea5bdc-616c-4de9-a7ac-8bf0b2b13e35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:55.742818 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ea5bdc-616c-4de9-a7ac-8bf0b2b13e35 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:55.743201 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ea5bdc-616c-4de9-a7ac-8bf0b2b13e35 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:55.743605 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ea5bdc-616c-4de9-a7ac-8bf0b2b13e35 None admin] PATH_INFO: `/v3/groups/3dad4633ec274cfb81918f0a9e56bead` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:55.744797 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ea5bdc-616c-4de9-a7ac-8bf0b2b13e35 None admin] RBAC: Authorizing `identity:get_group(group_id=3dad4633ec274cfb81918f0a9e56bead)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:55.746317 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ea5bdc-616c-4de9-a7ac-8bf0b2b13e35 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:55.752904 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 217/431] 10.222.0.18 () {56 vars in 1180 bytes} [Wed Aug 24 17:07:55 2022] GET /identity/v3/groups/3dad4633ec274cfb81918f0a9e56bead => generated 226 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:07:55.752904 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ea5bdc-616c-4de9-a7ac-8bf0b2b13e35 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:55.785246 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f184bb-c21a-4746-b844-3aab3a47f396 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:55.822810 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55d0e9e-416d-4372-b50e-5db8129965f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:55.824543 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55d0e9e-416d-4372-b50e-5db8129965f7 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:55.825281 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55d0e9e-416d-4372-b50e-5db8129965f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:55.825729 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55d0e9e-416d-4372-b50e-5db8129965f7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:55.826764 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55d0e9e-416d-4372-b50e-5db8129965f7 None admin] RBAC: Adding query filter params (group.id=3dad4633ec274cfb81918f0a9e56bead, role.id=60dc7e16aee746d5a3e8aa27d7d8525c, scope.project.id=9161a24523d545a8939fb56c3c930122) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:07:55.828000 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55d0e9e-416d-4372-b50e-5db8129965f7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:55.829588 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55d0e9e-416d-4372-b50e-5db8129965f7 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:55.837637 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 215/432] 10.222.0.18 () {56 vars in 1376 bytes} [Wed Aug 24 17:07:55 2022] GET /identity/v3/role_assignments?group.id=3dad4633ec274cfb81918f0a9e56bead&scope.project.id=9161a24523d545a8939fb56c3c930122&role.id=60dc7e16aee746d5a3e8aa27d7d8525c => generated 611 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:07:55.839582 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2942b1c-5a55-4081-aaa0-f56f93a9d9c5 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:55.840269 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2942b1c-5a55-4081-aaa0-f56f93a9d9c5 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:55.840688 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2942b1c-5a55-4081-aaa0-f56f93a9d9c5 None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:55.843052 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 216/433] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:07:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:07:55.844141 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2942b1c-5a55-4081-aaa0-f56f93a9d9c5 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:55.863939 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ea5bdc-616c-4de9-a7ac-8bf0b2b13e35 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:55.881720 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:55.905031 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa66fe33-7627-486f-ade9-75589df629f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:55.905685 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa66fe33-7627-486f-ade9-75589df629f9 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:55.905971 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa66fe33-7627-486f-ade9-75589df629f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:55.906250 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa66fe33-7627-486f-ade9-75589df629f9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:55.919743 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa66fe33-7627-486f-ade9-75589df629f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:07:55.926076 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-fa66fe33-7627-486f-ade9-75589df629f9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:07:55.926076 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:07:55.926076 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:07:55.926076 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:07:55.926076 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:07:55.926076 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:07:55.926076 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:07:55.926076 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:07:55.926076 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:07:55.926076 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:07:55.926076 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:07:55.926076 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:07:55.926076 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:07:55.926076 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:07:55.926076 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:07:55.926076 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:07:55.927368 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:07:55.927368 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:07:55.927368 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:07:55.927368 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:07:55.927368 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:07:55.927368 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:07:55.927368 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:07:55.927368 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:07:55.927368 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:07:55.927368 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:07:55.927368 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:07:55.927368 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application with Lock( Aug 24 17:07:55.927368 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:07:55.927368 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:07:55.927368 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:07:55.927368 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:07:55.928437 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:07:55.928437 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:07:55.928437 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:07:55.928437 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:07:55.928437 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:07:55.928437 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:07:55.928437 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:07:55.928437 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:07:55.928437 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:07:55.928437 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:07:55.928437 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 17:07:55.928437 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:07:55.932166 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 218/434] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 24 17:07:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:07:55.933680 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4740c6-d015-42a8-a17f-d7eae0c607b7 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:55.933976 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4740c6-d015-42a8-a17f-d7eae0c607b7 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:55.934252 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4740c6-d015-42a8-a17f-d7eae0c607b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:55.954118 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-ad4740c6-d015-42a8-a17f-d7eae0c607b7 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:07:55.955572 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-ad4740c6-d015-42a8-a17f-d7eae0c607b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:07:55.984175 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2942b1c-5a55-4081-aaa0-f56f93a9d9c5 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:55.993110 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:56.029532 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836064ba-d536-47b5-8217-d73200736928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:56.030949 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836064ba-d536-47b5-8217-d73200736928 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:56.031407 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836064ba-d536-47b5-8217-d73200736928 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:56.031877 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836064ba-d536-47b5-8217-d73200736928 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:56.032787 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836064ba-d536-47b5-8217-d73200736928 None admin] RBAC: Adding query filter params (group.id=3dad4633ec274cfb81918f0a9e56bead, role.id=af19a7a2fae5422da5bd7daea4bbfccf, scope.project.id=9161a24523d545a8939fb56c3c930122) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:07:56.033096 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836064ba-d536-47b5-8217-d73200736928 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:56.035092 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836064ba-d536-47b5-8217-d73200736928 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:56.042467 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 217/435] 10.222.0.18 () {56 vars in 1376 bytes} [Wed Aug 24 17:07:55 2022] GET /identity/v3/role_assignments?group.id=3dad4633ec274cfb81918f0a9e56bead&scope.project.id=9161a24523d545a8939fb56c3c930122&role.id=af19a7a2fae5422da5bd7daea4bbfccf => generated 611 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:56.042467 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836064ba-d536-47b5-8217-d73200736928 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:56.058809 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 219/436] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:07:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:07:56.171076 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836064ba-d536-47b5-8217-d73200736928 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:56.208660 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63f1d30-f31b-419f-96c6-81d1610b2893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:56.209518 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63f1d30-f31b-419f-96c6-81d1610b2893 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:56.209805 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63f1d30-f31b-419f-96c6-81d1610b2893 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:56.210078 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63f1d30-f31b-419f-96c6-81d1610b2893 None admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:56.210748 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63f1d30-f31b-419f-96c6-81d1610b2893 None admin] RBAC: Adding query filter params (name=member) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:07:56.211079 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63f1d30-f31b-419f-96c6-81d1610b2893 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:56.212565 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63f1d30-f31b-419f-96c6-81d1610b2893 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:56.222297 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 218/437] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Aug 24 17:07:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:56.227755 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4740c6-d015-42a8-a17f-d7eae0c607b7 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:56.339559 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4740c6-d015-42a8-a17f-d7eae0c607b7 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:56.342619 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:56.359656 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63f1d30-f31b-419f-96c6-81d1610b2893 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:56.385920 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4abdaf-cbb8-4b2d-a32d-842243521965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:56.386548 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4abdaf-cbb8-4b2d-a32d-842243521965 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:56.386825 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4abdaf-cbb8-4b2d-a32d-842243521965 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:56.387286 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4abdaf-cbb8-4b2d-a32d-842243521965 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:56.412923 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:56.477222 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4abdaf-cbb8-4b2d-a32d-842243521965 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:56.478592 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4abdaf-cbb8-4b2d-a32d-842243521965 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:56.504428 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63f1d30-f31b-419f-96c6-81d1610b2893 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:56.548669 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb63aea-ce55-466d-aa35-a3cac0719a46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:56.549325 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb63aea-ce55-466d-aa35-a3cac0719a46 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:56.549611 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb63aea-ce55-466d-aa35-a3cac0719a46 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:56.549886 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb63aea-ce55-466d-aa35-a3cac0719a46 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:56.564394 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb63aea-ce55-466d-aa35-a3cac0719a46 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:07:56.569756 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-adb63aea-ce55-466d-aa35-a3cac0719a46 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 17:07:56.569756 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:07:56.569756 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:07:56.569756 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:07:56.569756 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:07:56.569756 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:07:56.569756 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:07:56.569756 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:07:56.569756 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:07:56.569756 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:07:56.569756 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:07:56.569756 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:07:56.569756 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:07:56.569756 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:07:56.569756 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:07:56.569756 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:07:56.570665 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:07:56.570665 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:07:56.570665 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:07:56.570665 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:07:56.570665 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:07:56.570665 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:07:56.570665 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:07:56.570665 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:07:56.570665 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:07:56.570665 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:07:56.570665 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:07:56.570665 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 24 17:07:56.570665 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:07:56.570665 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:07:56.570665 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:07:56.570665 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:07:56.571589 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:07:56.571589 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:07:56.571589 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:07:56.571589 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:07:56.571589 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:07:56.571589 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:07:56.571589 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:07:56.571589 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:07:56.571589 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:07:56.571589 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:07:56.571589 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 17:07:56.571589 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:07:56.571589 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 219/438] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Aug 24 17:07:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:07:56.572241 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea8da34-8d9a-4c4d-8986-5398bf0850e8 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:56.572318 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea8da34-8d9a-4c4d-8986-5398bf0850e8 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:56.572740 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea8da34-8d9a-4c4d-8986-5398bf0850e8 None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:56.573757 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 220/439] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:07:56 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:07:56.578533 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea8da34-8d9a-4c4d-8986-5398bf0850e8 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:56.593273 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 220/440] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 24 17:07:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 17:07:56.594379 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ae1487-54a9-4497-b6fb-b8a1437ecb3e None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:56.594666 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ae1487-54a9-4497-b6fb-b8a1437ecb3e None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:56.594958 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ae1487-54a9-4497-b6fb-b8a1437ecb3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:56.602144 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:56.614553 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-e1ae1487-54a9-4497-b6fb-b8a1437ecb3e None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:07:56.615795 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-e1ae1487-54a9-4497-b6fb-b8a1437ecb3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:07:56.685538 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 221/441] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:07:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:07:56.686373 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ae1487-54a9-4497-b6fb-b8a1437ecb3e None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:56.690492 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:56.699442 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea8da34-8d9a-4c4d-8986-5398bf0850e8 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:56.735161 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02eba94-7114-40bc-b640-165252c97d60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:56.736109 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02eba94-7114-40bc-b640-165252c97d60 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:56.736109 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02eba94-7114-40bc-b640-165252c97d60 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:56.736278 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02eba94-7114-40bc-b640-165252c97d60 None admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:56.736843 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02eba94-7114-40bc-b640-165252c97d60 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:07:56.737141 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02eba94-7114-40bc-b640-165252c97d60 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:56.738515 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02eba94-7114-40bc-b640-165252c97d60 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:56.746795 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 221/442] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 24 17:07:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:07:56.752822 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02eba94-7114-40bc-b640-165252c97d60 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:56.804544 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ae1487-54a9-4497-b6fb-b8a1437ecb3e None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:56.839501 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a353d045-c1da-4864-8c08-e9bc17cf3af0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:56.840337 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a353d045-c1da-4864-8c08-e9bc17cf3af0 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:56.840449 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a353d045-c1da-4864-8c08-e9bc17cf3af0 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:56.840667 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a353d045-c1da-4864-8c08-e9bc17cf3af0 None admin] PATH_INFO: `/v3/projects/94586413811643bb999468e84b659403` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:56.842293 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a353d045-c1da-4864-8c08-e9bc17cf3af0 None admin] RBAC: Authorizing `identity:get_project(project_id=94586413811643bb999468e84b659403)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:56.843778 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a353d045-c1da-4864-8c08-e9bc17cf3af0 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:56.845413 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 222/443] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 24 17:07:56 2022] GET /identity/v3/projects/94586413811643bb999468e84b659403 => generated 302 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:56.850724 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a353d045-c1da-4864-8c08-e9bc17cf3af0 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:56.853538 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:56.865068 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02eba94-7114-40bc-b640-165252c97d60 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:56.901889 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fda4d4c-b85d-41ee-86ab-07549d9c4b32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:56.902489 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fda4d4c-b85d-41ee-86ab-07549d9c4b32 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:56.902832 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fda4d4c-b85d-41ee-86ab-07549d9c4b32 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:56.903118 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fda4d4c-b85d-41ee-86ab-07549d9c4b32 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:56.928591 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:56.961445 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a353d045-c1da-4864-8c08-e9bc17cf3af0 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:56.972887 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fda4d4c-b85d-41ee-86ab-07549d9c4b32 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:56.974330 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fda4d4c-b85d-41ee-86ab-07549d9c4b32 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:56.996046 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2c8cb1-ef68-4a79-a461-042796e7923e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:56.996601 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2c8cb1-ef68-4a79-a461-042796e7923e None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:56.996878 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2c8cb1-ef68-4a79-a461-042796e7923e None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:56.997145 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2c8cb1-ef68-4a79-a461-042796e7923e None admin] PATH_INFO: `/v3/groups/3dad4633ec274cfb81918f0a9e56bead` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:56.998256 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2c8cb1-ef68-4a79-a461-042796e7923e None admin] RBAC: Authorizing `identity:get_group(group_id=3dad4633ec274cfb81918f0a9e56bead)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:56.999707 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2c8cb1-ef68-4a79-a461-042796e7923e None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:57.001668 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 223/444] 10.222.0.18 () {56 vars in 1180 bytes} [Wed Aug 24 17:07:56 2022] GET /identity/v3/groups/3dad4633ec274cfb81918f0a9e56bead => generated 226 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:07:57.006880 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2c8cb1-ef68-4a79-a461-042796e7923e None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:57.071534 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:57.071951 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 222/445] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 24 17:07:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 17:07:57.071951 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fda4d4c-b85d-41ee-86ab-07549d9c4b32 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:57.105517 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2c8cb1-ef68-4a79-a461-042796e7923e None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:57.128144 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:57.140589 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bdeabb-a7b9-43a1-9e6c-7d83405182e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:57.141207 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bdeabb-a7b9-43a1-9e6c-7d83405182e1 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:57.141486 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bdeabb-a7b9-43a1-9e6c-7d83405182e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:57.141756 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bdeabb-a7b9-43a1-9e6c-7d83405182e1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:57.151759 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bdeabb-a7b9-43a1-9e6c-7d83405182e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:07:57.157420 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-43bdeabb-a7b9-43a1-9e6c-7d83405182e1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:07:57.157420 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:07:57.157420 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:07:57.157420 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:07:57.157420 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:07:57.157420 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:07:57.157420 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:07:57.157420 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:07:57.157420 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:07:57.157420 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:07:57.157420 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:07:57.157420 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:07:57.157420 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:07:57.157420 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:07:57.157420 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:07:57.157420 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:07:57.158517 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:07:57.158517 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:07:57.158517 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:07:57.158517 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:07:57.158517 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:07:57.158517 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:07:57.158517 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:07:57.158517 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:07:57.158517 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:07:57.158517 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:07:57.158517 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:07:57.158517 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application with Lock( Aug 24 17:07:57.158517 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:07:57.158517 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:07:57.158517 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:07:57.158517 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:07:57.159551 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:07:57.159551 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:07:57.159551 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:07:57.159551 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:07:57.159551 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:07:57.159551 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:07:57.159551 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:07:57.159551 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:07:57.159551 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:07:57.159551 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:07:57.159551 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:07:57.159551 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:07:57.159551 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 224/446] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Aug 24 17:07:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:07:57.159551 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bdeabb-a7b9-43a1-9e6c-7d83405182e1 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:57.177531 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fda4d4c-b85d-41ee-86ab-07549d9c4b32 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:57.209985 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd457301-41b6-473d-95ef-cf3238e1bf18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:57.210582 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd457301-41b6-473d-95ef-cf3238e1bf18 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:57.210859 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd457301-41b6-473d-95ef-cf3238e1bf18 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:57.211157 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd457301-41b6-473d-95ef-cf3238e1bf18 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:57.211864 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd457301-41b6-473d-95ef-cf3238e1bf18 None admin] RBAC: Adding query filter params (group.id=3dad4633ec274cfb81918f0a9e56bead, role.id=60dc7e16aee746d5a3e8aa27d7d8525c, scope.project.id=94586413811643bb999468e84b659403) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:07:57.212163 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd457301-41b6-473d-95ef-cf3238e1bf18 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:57.213629 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd457301-41b6-473d-95ef-cf3238e1bf18 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:57.219821 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 223/447] 10.222.0.18 () {56 vars in 1376 bytes} [Wed Aug 24 17:07:57 2022] GET /identity/v3/role_assignments?group.id=3dad4633ec274cfb81918f0a9e56bead&scope.project.id=94586413811643bb999468e84b659403&role.id=60dc7e16aee746d5a3e8aa27d7d8525c => generated 611 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:57.220583 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd457301-41b6-473d-95ef-cf3238e1bf18 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:57.276112 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bdeabb-a7b9-43a1-9e6c-7d83405182e1 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:57.307248 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:57.313277 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d9f3be-a807-4eb7-9442-99848c81e391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:57.313917 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d9f3be-a807-4eb7-9442-99848c81e391 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:57.314192 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d9f3be-a807-4eb7-9442-99848c81e391 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:57.316870 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d9f3be-a807-4eb7-9442-99848c81e391 None admin] PATH_INFO: `/v3/projects/94586413811643bb999468e84b659403` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:57.316870 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d9f3be-a807-4eb7-9442-99848c81e391 None admin] RBAC: Authorizing `identity:get_project(project_id=94586413811643bb999468e84b659403)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:57.317184 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d9f3be-a807-4eb7-9442-99848c81e391 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:57.318809 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 225/448] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 24 17:07:57 2022] GET /identity/v3/projects/94586413811643bb999468e84b659403 => generated 302 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:57.324456 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d9f3be-a807-4eb7-9442-99848c81e391 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:57.339731 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd457301-41b6-473d-95ef-cf3238e1bf18 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:57.358983 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:57.376545 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55be67f1-6160-40fa-a894-42725f38fdea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:57.377022 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55be67f1-6160-40fa-a894-42725f38fdea None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:57.377532 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55be67f1-6160-40fa-a894-42725f38fdea None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:57.377532 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55be67f1-6160-40fa-a894-42725f38fdea None admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:57.378197 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55be67f1-6160-40fa-a894-42725f38fdea None admin] RBAC: Adding query filter params (name=admin) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:07:57.378496 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55be67f1-6160-40fa-a894-42725f38fdea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:57.379905 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55be67f1-6160-40fa-a894-42725f38fdea None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:57.390738 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 224/449] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 24 17:07:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:57.396031 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55be67f1-6160-40fa-a894-42725f38fdea None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:57.454083 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d9f3be-a807-4eb7-9442-99848c81e391 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:57.497719 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7c574f-1741-4a36-b7e4-94c0de6d3fa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:57.498749 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7c574f-1741-4a36-b7e4-94c0de6d3fa0 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:57.498749 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7c574f-1741-4a36-b7e4-94c0de6d3fa0 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:57.499008 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7c574f-1741-4a36-b7e4-94c0de6d3fa0 None admin] PATH_INFO: `/v3/groups/3dad4633ec274cfb81918f0a9e56bead` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:57.500009 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7c574f-1741-4a36-b7e4-94c0de6d3fa0 None admin] RBAC: Authorizing `identity:get_group(group_id=3dad4633ec274cfb81918f0a9e56bead)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:57.501694 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7c574f-1741-4a36-b7e4-94c0de6d3fa0 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:57.503284 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 226/450] 10.222.0.18 () {56 vars in 1180 bytes} [Wed Aug 24 17:07:57 2022] GET /identity/v3/groups/3dad4633ec274cfb81918f0a9e56bead => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:07:57.509065 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7c574f-1741-4a36-b7e4-94c0de6d3fa0 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:57.518771 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55be67f1-6160-40fa-a894-42725f38fdea None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:57.568823 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ad6943-8384-464c-9c5b-a01f47dd0b9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:57.569376 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ad6943-8384-464c-9c5b-a01f47dd0b9c None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:57.570458 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ad6943-8384-464c-9c5b-a01f47dd0b9c None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:57.570458 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ad6943-8384-464c-9c5b-a01f47dd0b9c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:57.592771 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:57.619869 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:57.627079 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7c574f-1741-4a36-b7e4-94c0de6d3fa0 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:57.646262 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ad6943-8384-464c-9c5b-a01f47dd0b9c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:57.648141 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ad6943-8384-464c-9c5b-a01f47dd0b9c None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:57.662684 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30094b2e-d92a-4b12-9dda-5bd2781851f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:57.663325 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30094b2e-d92a-4b12-9dda-5bd2781851f7 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:57.663604 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30094b2e-d92a-4b12-9dda-5bd2781851f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:57.663875 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30094b2e-d92a-4b12-9dda-5bd2781851f7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:57.664602 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30094b2e-d92a-4b12-9dda-5bd2781851f7 None admin] RBAC: Adding query filter params (group.id=3dad4633ec274cfb81918f0a9e56bead, role.id=af19a7a2fae5422da5bd7daea4bbfccf, scope.project.id=94586413811643bb999468e84b659403) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:07:57.664912 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30094b2e-d92a-4b12-9dda-5bd2781851f7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:57.666344 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30094b2e-d92a-4b12-9dda-5bd2781851f7 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:57.672154 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 227/451] 10.222.0.18 () {56 vars in 1376 bytes} [Wed Aug 24 17:07:57 2022] GET /identity/v3/role_assignments?group.id=3dad4633ec274cfb81918f0a9e56bead&scope.project.id=94586413811643bb999468e84b659403&role.id=af19a7a2fae5422da5bd7daea4bbfccf => generated 611 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:07:57.754559 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 225/452] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 24 17:07:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 17:07:57.760895 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30094b2e-d92a-4b12-9dda-5bd2781851f7 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:57.870202 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30094b2e-d92a-4b12-9dda-5bd2781851f7 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:57.900087 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:57.905934 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456e158f-4d09-4c10-b795-263bc6749c5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:57.911018 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456e158f-4d09-4c10-b795-263bc6749c5d None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:57.911018 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456e158f-4d09-4c10-b795-263bc6749c5d None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:57.911018 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456e158f-4d09-4c10-b795-263bc6749c5d None admin] PATH_INFO: `/v3/projects/68a3eec6caa5424ea6bb39d5d6e46c2f` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:57.911018 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456e158f-4d09-4c10-b795-263bc6749c5d None admin] RBAC: Authorizing `identity:get_project(project_id=68a3eec6caa5424ea6bb39d5d6e46c2f)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:57.911018 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456e158f-4d09-4c10-b795-263bc6749c5d None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:57.911641 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 228/453] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 24 17:07:57 2022] GET /identity/v3/projects/68a3eec6caa5424ea6bb39d5d6e46c2f => generated 344 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:07:57.917262 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ad6943-8384-464c-9c5b-a01f47dd0b9c None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:58.002417 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:07:58.021219 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ad6943-8384-464c-9c5b-a01f47dd0b9c None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:58.054471 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7702d6-0150-4e7a-b50c-5ccc3f647d1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:58.055116 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7702d6-0150-4e7a-b50c-5ccc3f647d1d None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:58.055400 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7702d6-0150-4e7a-b50c-5ccc3f647d1d None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:58.055697 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7702d6-0150-4e7a-b50c-5ccc3f647d1d None admin] PATH_INFO: `/v3/groups/0630417565324d3e96df34a8ff800e50` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:58.056783 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7702d6-0150-4e7a-b50c-5ccc3f647d1d None admin] RBAC: Authorizing `identity:get_group(group_id=0630417565324d3e96df34a8ff800e50)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:58.058206 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7702d6-0150-4e7a-b50c-5ccc3f647d1d None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:58.060424 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 226/454] 10.222.0.18 () {56 vars in 1180 bytes} [Wed Aug 24 17:07:57 2022] GET /identity/v3/groups/0630417565324d3e96df34a8ff800e50 => generated 229 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:07:58.066430 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456e158f-4d09-4c10-b795-263bc6749c5d None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:07:58.160593 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456e158f-4d09-4c10-b795-263bc6749c5d None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:07:58.192641 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75603de6-7cb5-4f7e-9c7c-53c656ff9a6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:07:58.193218 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75603de6-7cb5-4f7e-9c7c-53c656ff9a6f None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:07:58.193496 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75603de6-7cb5-4f7e-9c7c-53c656ff9a6f None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:07:58.193763 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75603de6-7cb5-4f7e-9c7c-53c656ff9a6f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:07:58.194473 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75603de6-7cb5-4f7e-9c7c-53c656ff9a6f None admin] RBAC: Adding query filter params (group.id=0630417565324d3e96df34a8ff800e50, role.id=c5346bc54abf445f9900edf94264c850, scope.project.id=68a3eec6caa5424ea6bb39d5d6e46c2f) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:07:58.194777 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75603de6-7cb5-4f7e-9c7c-53c656ff9a6f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:07:58.196220 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75603de6-7cb5-4f7e-9c7c-53c656ff9a6f None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:07:58.201659 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 229/455] 10.222.0.18 () {56 vars in 1376 bytes} [Wed Aug 24 17:07:58 2022] GET /identity/v3/role_assignments?group.id=0630417565324d3e96df34a8ff800e50&scope.project.id=68a3eec6caa5424ea6bb39d5d6e46c2f&role.id=c5346bc54abf445f9900edf94264c850 => generated 611 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:00.445712 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae04da2-33c9-4b09-91f7-2bce5b2f435b None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:00.446251 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae04da2-33c9-4b09-91f7-2bce5b2f435b None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:00.446251 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae04da2-33c9-4b09-91f7-2bce5b2f435b None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:00.447872 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 227/456] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:08:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:08:00.453612 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800627f9-5394-4bd0-947d-75808eb61991 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:00.454103 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800627f9-5394-4bd0-947d-75808eb61991 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:00.454187 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800627f9-5394-4bd0-947d-75808eb61991 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:00.476220 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-800627f9-5394-4bd0-947d-75808eb61991 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:08:00.477404 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-800627f9-5394-4bd0-947d-75808eb61991 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:08:00.555268 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:00.568219 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 230/457] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:08:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 17:08:00.663189 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beefca0a-a6b0-40a8-b130-90653628365c None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:00.663547 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beefca0a-a6b0-40a8-b130-90653628365c None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:00.663737 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beefca0a-a6b0-40a8-b130-90653628365c None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:00.666011 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 228/458] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:08:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:08:00.671176 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a446712e-c9f2-4f4d-a5d9-317062cfc4b6 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:00.671459 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a446712e-c9f2-4f4d-a5d9-317062cfc4b6 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:00.671730 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a446712e-c9f2-4f4d-a5d9-317062cfc4b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:00.695999 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-a446712e-c9f2-4f4d-a5d9-317062cfc4b6 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:08:00.697120 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-a446712e-c9f2-4f4d-a5d9-317062cfc4b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:08:00.788441 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 231/459] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:08:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:08:00.924506 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beefca0a-a6b0-40a8-b130-90653628365c None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:01.064434 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beefca0a-a6b0-40a8-b130-90653628365c None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:01.113930 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf92a01-b68f-43f1-8062-d4a4b3b7707f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:01.114971 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf92a01-b68f-43f1-8062-d4a4b3b7707f None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:01.115224 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b7f0d2-7517-4bf1-b6fa-246fbaf0a4a9 None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:01.115511 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b7f0d2-7517-4bf1-b6fa-246fbaf0a4a9 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:01.115786 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b7f0d2-7517-4bf1-b6fa-246fbaf0a4a9 None None] PATH_INFO: `/` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:01.116093 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf92a01-b68f-43f1-8062-d4a4b3b7707f None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:01.116566 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf92a01-b68f-43f1-8062-d4a4b3b7707f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:01.117107 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 232/460] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:08:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 17:08:01.131343 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06d0663-26b2-4184-afec-4a3c471edda6 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:01.131511 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06d0663-26b2-4184-afec-4a3c471edda6 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:01.131781 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06d0663-26b2-4184-afec-4a3c471edda6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:01.132947 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf92a01-b68f-43f1-8062-d4a4b3b7707f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:01.134746 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf92a01-b68f-43f1-8062-d4a4b3b7707f None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:01.150399 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-6cf92a01-b68f-43f1-8062-d4a4b3b7707f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:08:01.150399 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:01.150399 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:01.150399 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:01.150399 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:01.150399 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:01.150399 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:01.150399 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:01.150399 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:01.150399 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:01.150399 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:01.150399 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:01.150399 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:08:01.150399 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:08:01.150399 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:08:01.150399 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:08:01.151448 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:01.151448 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:01.151448 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:01.151448 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:01.151448 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:01.151448 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:01.151448 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:01.151448 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:01.151448 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:01.151448 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:01.151448 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:01.151448 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:01.151448 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:01.151448 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:01.151448 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:01.151448 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:01.151448 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:08:01.152331 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:08:01.152331 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:08:01.152331 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:08:01.154626 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 229/461] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 24 17:08:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:01.158926 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:01.160741 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf92a01-b68f-43f1-8062-d4a4b3b7707f None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:01.164467 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-a06d0663-26b2-4184-afec-4a3c471edda6 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:08:01.165654 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-a06d0663-26b2-4184-afec-4a3c471edda6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:08:01.265038 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 233/462] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:08:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:08:01.271090 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e322616-6f04-4d33-b109-528e578a1bc6 None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:01.271090 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e322616-6f04-4d33-b109-528e578a1bc6 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:01.271090 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e322616-6f04-4d33-b109-528e578a1bc6 None None] PATH_INFO: `/` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:01.274792 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 234/463] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:08:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:08:01.277825 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e39051-ffdf-423d-bf5d-2c5e15f310f3 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:01.278104 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e39051-ffdf-423d-bf5d-2c5e15f310f3 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:01.278989 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e39051-ffdf-423d-bf5d-2c5e15f310f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:01.289417 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf92a01-b68f-43f1-8062-d4a4b3b7707f None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:01.308650 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-76e39051-ffdf-423d-bf5d-2c5e15f310f3 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:08:01.310325 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-76e39051-ffdf-423d-bf5d-2c5e15f310f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:08:01.328358 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c8acf5-86cd-4e0c-a448-7eee211bbf38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:01.329067 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c8acf5-86cd-4e0c-a448-7eee211bbf38 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:01.329349 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c8acf5-86cd-4e0c-a448-7eee211bbf38 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:01.329618 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c8acf5-86cd-4e0c-a448-7eee211bbf38 None admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:01.330254 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c8acf5-86cd-4e0c-a448-7eee211bbf38 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:01.330553 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c8acf5-86cd-4e0c-a448-7eee211bbf38 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:01.332169 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c8acf5-86cd-4e0c-a448-7eee211bbf38 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:01.345522 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 230/464] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 24 17:08:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:01.346459 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c8acf5-86cd-4e0c-a448-7eee211bbf38 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:01.369746 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:01.399035 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 235/465] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:08:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:08:01.400107 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e39051-ffdf-423d-bf5d-2c5e15f310f3 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:01.423200 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:01.492791 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c8acf5-86cd-4e0c-a448-7eee211bbf38 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:01.562855 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07480484-c654-40ce-a855-c27c1519dc2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:01.564727 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e39051-ffdf-423d-bf5d-2c5e15f310f3 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:01.566570 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07480484-c654-40ce-a855-c27c1519dc2a None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:01.566843 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07480484-c654-40ce-a855-c27c1519dc2a None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:01.567132 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07480484-c654-40ce-a855-c27c1519dc2a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:01.587423 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07480484-c654-40ce-a855-c27c1519dc2a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:01.588814 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07480484-c654-40ce-a855-c27c1519dc2a None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:01.616411 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-07480484-c654-40ce-a855-c27c1519dc2a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:08:01.616411 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:01.616411 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:01.616411 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:01.616411 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:01.616411 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:01.616411 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:01.616411 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:01.616411 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:01.616411 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:01.616411 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:01.616411 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:01.616411 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:08:01.616411 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:08:01.616411 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:08:01.616411 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:08:01.617393 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:01.617393 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:01.617393 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:01.617393 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:01.617393 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:01.617393 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:01.617393 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:01.617393 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:01.617393 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:01.617393 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:01.617393 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:01.617393 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:01.617393 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:01.617393 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:01.617393 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:01.617393 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:01.617393 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:08:01.618536 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:08:01.618536 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:08:01.618536 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:08:01.618536 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 231/466] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 24 17:08:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:01.618777 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07480484-c654-40ce-a855-c27c1519dc2a None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:01.628991 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9884a032-4a20-4182-98fb-9c1c7ad2568f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:01.630573 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9884a032-4a20-4182-98fb-9c1c7ad2568f None admin] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:01.630573 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9884a032-4a20-4182-98fb-9c1c7ad2568f None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:01.630573 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9884a032-4a20-4182-98fb-9c1c7ad2568f None admin] PATH_INFO: `/v3/users` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:01.631099 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9884a032-4a20-4182-98fb-9c1c7ad2568f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:01.633058 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9884a032-4a20-4182-98fb-9c1c7ad2568f None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:01.737595 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07480484-c654-40ce-a855-c27c1519dc2a None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:01.738695 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 236/467] 10.222.0.18 () {60 vars in 1132 bytes} [Wed Aug 24 17:08:01 2022] POST /identity/v3/users => generated 246 bytes in 339 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:08:01.739628 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9884a032-4a20-4182-98fb-9c1c7ad2568f None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:01.765550 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:01.790724 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:01.797617 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5d4292-3756-4f7c-b2bb-5944319bd05e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:01.798272 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5d4292-3756-4f7c-b2bb-5944319bd05e None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:01.798549 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5d4292-3756-4f7c-b2bb-5944319bd05e None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:01.798819 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5d4292-3756-4f7c-b2bb-5944319bd05e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:01.814557 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5d4292-3756-4f7c-b2bb-5944319bd05e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:01.817082 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5d4292-3756-4f7c-b2bb-5944319bd05e None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:01.837419 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-5b5d4292-3756-4f7c-b2bb-5944319bd05e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:08:01.837419 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:01.837419 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:01.837419 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:01.837419 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:01.837419 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:01.837419 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:01.837419 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:01.837419 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:01.837419 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:01.837419 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:01.837419 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:01.837419 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:08:01.837419 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:08:01.837419 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:08:01.837419 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:08:01.838431 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:01.838431 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:01.838431 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:01.838431 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:01.838431 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:01.838431 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:01.838431 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:01.838431 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:01.838431 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:01.838431 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:01.838431 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:01.838431 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:01.838431 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:01.838431 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:01.838431 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:01.838431 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:01.838431 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:08:01.839703 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:08:01.839703 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:08:01.839703 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:08:01.839703 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 232/468] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 24 17:08:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:01.843978 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5d4292-3756-4f7c-b2bb-5944319bd05e None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:01.881571 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9884a032-4a20-4182-98fb-9c1c7ad2568f None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:01.928665 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cab7be7-1f9b-4952-8c4d-626ee56217e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:01.929681 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cab7be7-1f9b-4952-8c4d-626ee56217e3 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:01.930165 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cab7be7-1f9b-4952-8c4d-626ee56217e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:01.930644 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cab7be7-1f9b-4952-8c4d-626ee56217e3 None admin] PATH_INFO: `/v3/domains` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:01.931509 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cab7be7-1f9b-4952-8c4d-626ee56217e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:01.932015 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cab7be7-1f9b-4952-8c4d-626ee56217e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:01.933634 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cab7be7-1f9b-4952-8c4d-626ee56217e3 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:01.947741 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 237/469] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 24 17:08:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:01.948383 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cab7be7-1f9b-4952-8c4d-626ee56217e3 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:01.966932 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5d4292-3756-4f7c-b2bb-5944319bd05e None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:02.004161 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c878187-f93a-4c20-b895-1c40cfc57243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:02.004793 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c878187-f93a-4c20-b895-1c40cfc57243 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:02.005111 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c878187-f93a-4c20-b895-1c40cfc57243 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:02.005428 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c878187-f93a-4c20-b895-1c40cfc57243 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:02.016963 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c878187-f93a-4c20-b895-1c40cfc57243 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:02.018426 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c878187-f93a-4c20-b895-1c40cfc57243 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:02.032987 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-9c878187-f93a-4c20-b895-1c40cfc57243 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:08:02.032987 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:02.032987 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:02.032987 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:02.032987 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:02.032987 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:02.032987 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:02.032987 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:02.032987 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:02.032987 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:02.032987 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:02.032987 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:02.032987 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:08:02.032987 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:08:02.032987 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:08:02.032987 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:08:02.033911 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:02.033911 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:02.033911 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:02.033911 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:02.033911 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:02.033911 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:02.033911 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:02.033911 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:02.033911 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:02.033911 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:02.033911 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:02.033911 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:02.033911 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:02.033911 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:02.033911 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:02.033911 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:02.033911 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:08:02.034735 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:08:02.034735 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:08:02.034735 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:08:02.039036 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 233/470] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 24 17:08:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:02.039964 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c878187-f93a-4c20-b895-1c40cfc57243 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:02.059003 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:02.079600 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:02.083546 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cab7be7-1f9b-4952-8c4d-626ee56217e3 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:02.121116 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bda45d-3acf-4f48-a207-b8fe2cc4a5c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:02.121645 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bda45d-3acf-4f48-a207-b8fe2cc4a5c4 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:02.121941 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bda45d-3acf-4f48-a207-b8fe2cc4a5c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:02.123152 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bda45d-3acf-4f48-a207-b8fe2cc4a5c4 None admin] PATH_INFO: `/v3/domains` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:02.123812 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bda45d-3acf-4f48-a207-b8fe2cc4a5c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:02.124108 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bda45d-3acf-4f48-a207-b8fe2cc4a5c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:02.125474 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bda45d-3acf-4f48-a207-b8fe2cc4a5c4 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:02.139314 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 238/471] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 24 17:08:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:02.140326 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bda45d-3acf-4f48-a207-b8fe2cc4a5c4 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:02.164660 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c878187-f93a-4c20-b895-1c40cfc57243 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:02.199559 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9261843c-b359-4530-bdac-ff5feb5e93b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:02.200178 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9261843c-b359-4530-bdac-ff5feb5e93b0 None admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:02.200993 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9261843c-b359-4530-bdac-ff5feb5e93b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:02.201319 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9261843c-b359-4530-bdac-ff5feb5e93b0 None admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:02.201947 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9261843c-b359-4530-bdac-ff5feb5e93b0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:02.203432 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9261843c-b359-4530-bdac-ff5feb5e93b0 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:02.243240 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bda45d-3acf-4f48-a207-b8fe2cc4a5c4 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:02.278883 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f998bdb-4774-49be-b662-eec2c41dd856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:02.279555 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f998bdb-4774-49be-b662-eec2c41dd856 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:02.279869 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f998bdb-4774-49be-b662-eec2c41dd856 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:02.280140 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f998bdb-4774-49be-b662-eec2c41dd856 None admin] PATH_INFO: `/v3/domains` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:02.280815 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f998bdb-4774-49be-b662-eec2c41dd856 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:02.281111 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f998bdb-4774-49be-b662-eec2c41dd856 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:02.282456 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f998bdb-4774-49be-b662-eec2c41dd856 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:02.293424 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 234/472] 10.222.0.18 () {60 vars in 1132 bytes} [Wed Aug 24 17:08:02 2022] POST /identity/v3/users => generated 248 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:08:02.294825 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 239/473] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 24 17:08:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:02.295808 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f998bdb-4774-49be-b662-eec2c41dd856 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:02.300210 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8389fc49-459f-43e3-992b-19f3f72a6122 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:02.300556 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8389fc49-459f-43e3-992b-19f3f72a6122 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:02.300844 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8389fc49-459f-43e3-992b-19f3f72a6122 None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:02.305274 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 235/474] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:08:02 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:08:02.305274 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8389fc49-459f-43e3-992b-19f3f72a6122 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:02.328442 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:02.403894 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:02.440878 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f998bdb-4774-49be-b662-eec2c41dd856 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:02.442740 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8389fc49-459f-43e3-992b-19f3f72a6122 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:02.483612 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92588ad3-b722-4507-8d18-8415cb20273a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:02.484226 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92588ad3-b722-4507-8d18-8415cb20273a None admin] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:02.484507 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92588ad3-b722-4507-8d18-8415cb20273a None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:02.485396 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92588ad3-b722-4507-8d18-8415cb20273a None admin] PATH_INFO: `/v3/users` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:02.485396 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92588ad3-b722-4507-8d18-8415cb20273a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:02.486824 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92588ad3-b722-4507-8d18-8415cb20273a None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:02.498448 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f5d0a1-8c19-4a20-a768-3997161ace33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:02.499893 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f5d0a1-8c19-4a20-a768-3997161ace33 None admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:02.500326 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f5d0a1-8c19-4a20-a768-3997161ace33 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:02.500863 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f5d0a1-8c19-4a20-a768-3997161ace33 None admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:02.501632 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f5d0a1-8c19-4a20-a768-3997161ace33 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:02.503360 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f5d0a1-8c19-4a20-a768-3997161ace33 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:02.594523 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 240/475] 10.222.0.18 () {60 vars in 1132 bytes} [Wed Aug 24 17:08:02 2022] POST /identity/v3/users => generated 249 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:08:02.595687 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b60a613-ed85-4030-8de9-f09cf3b9eeb2 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:02.595979 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b60a613-ed85-4030-8de9-f09cf3b9eeb2 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:02.596276 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b60a613-ed85-4030-8de9-f09cf3b9eeb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:02.616616 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-4b60a613-ed85-4030-8de9-f09cf3b9eeb2 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:08:02.618341 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-4b60a613-ed85-4030-8de9-f09cf3b9eeb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:08:02.627785 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 236/476] 10.222.0.18 () {60 vars in 1132 bytes} [Wed Aug 24 17:08:02 2022] POST /identity/v3/users => generated 248 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:08:02.720162 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 241/477] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:08:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:08:03.027216 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f5d0a1-8c19-4a20-a768-3997161ace33 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:03.147256 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f5d0a1-8c19-4a20-a768-3997161ace33 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:03.174839 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:03.187659 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bff0b5-777d-449c-a90d-bb16a07ffe77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:03.188347 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bff0b5-777d-449c-a90d-bb16a07ffe77 None admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:03.188639 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bff0b5-777d-449c-a90d-bb16a07ffe77 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:03.188916 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bff0b5-777d-449c-a90d-bb16a07ffe77 None admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:03.189585 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bff0b5-777d-449c-a90d-bb16a07ffe77 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:03.190984 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bff0b5-777d-449c-a90d-bb16a07ffe77 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:03.258553 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.sql.core [None req-f4bff0b5-777d-449c-a90d-bb16a07ffe77 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 24 17:08:03.258553 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 24 17:08:03.258553 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [parameters: {'id': '64b2132cece44ee9a5a72ddd60f8a191', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 24 17:08:03.258553 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81732) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 24 17:08:03.280547 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-f4bff0b5-777d-449c-a90d-bb16a07ffe77 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 24 17:08:03.280547 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:03.280547 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 17:08:03.280547 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 17:08:03.280547 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 17:08:03.280547 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 17:08:03.280547 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 17:08:03.280547 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 17:08:03.280547 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 17:08:03.280547 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application conn.query(q) Aug 24 17:08:03.280547 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 17:08:03.280547 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 17:08:03.280547 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 17:08:03.280547 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application result.read() Aug 24 17:08:03.280547 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 17:08:03.281478 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 17:08:03.281478 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 17:08:03.281478 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 17:08:03.281478 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 17:08:03.281478 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 17:08:03.281478 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 17:08:03.281478 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 17:08:03.281478 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 24 17:08:03.281478 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:08:03.281478 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 24 17:08:03.281478 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:08:03.281478 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:03.281478 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 24 17:08:03.281478 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 24 17:08:03.281478 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 24 17:08:03.281478 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return ref.to_dict() Aug 24 17:08:03.281478 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 17:08:03.281478 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application next(self.gen) Aug 24 17:08:03.282448 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 24 17:08:03.282448 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application yield resource Aug 24 17:08:03.282448 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 17:08:03.282448 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application next(self.gen) Aug 24 17:08:03.282448 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 24 17:08:03.282448 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application self.session.rollback() Aug 24 17:08:03.282448 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 24 17:08:03.282448 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application self.force_reraise() Aug 24 17:08:03.282448 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 24 17:08:03.282448 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise self.value Aug 24 17:08:03.282448 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 24 17:08:03.282448 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 24 17:08:03.282448 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 24 17:08:03.282448 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application session.commit() Aug 24 17:08:03.282448 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 24 17:08:03.282448 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 24 17:08:03.282448 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 24 17:08:03.283929 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application self._prepare_impl() Aug 24 17:08:03.283929 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 24 17:08:03.283929 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application self.session.flush() Aug 24 17:08:03.283929 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 24 17:08:03.283929 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application self._flush(objects) Aug 24 17:08:03.283929 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 24 17:08:03.283929 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 24 17:08:03.283929 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 24 17:08:03.283929 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application compat.raise_( Aug 24 17:08:03.283929 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 17:08:03.283929 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception Aug 24 17:08:03.283929 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 24 17:08:03.283929 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application flush_context.execute() Aug 24 17:08:03.283929 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 24 17:08:03.283929 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rec.execute(self) Aug 24 17:08:03.283929 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 24 17:08:03.283929 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 24 17:08:03.284822 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 24 17:08:03.284822 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 24 17:08:03.284822 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 24 17:08:03.284822 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 24 17:08:03.284822 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 24 17:08:03.284822 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 24 17:08:03.284822 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 24 17:08:03.284822 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 24 17:08:03.284822 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 24 17:08:03.284822 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 24 17:08:03.284822 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 24 17:08:03.284822 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 24 17:08:03.284822 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 24 17:08:03.284822 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 24 17:08:03.284822 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 17:08:03.284822 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception Aug 24 17:08:03.285614 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 17:08:03.285614 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 17:08:03.285614 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 17:08:03.285614 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 17:08:03.285614 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 17:08:03.285614 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 17:08:03.285614 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 17:08:03.285614 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application conn.query(q) Aug 24 17:08:03.285614 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 17:08:03.285614 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 17:08:03.285614 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 17:08:03.285614 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application result.read() Aug 24 17:08:03.285614 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 17:08:03.285614 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 17:08:03.285614 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 17:08:03.285614 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 17:08:03.287901 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 17:08:03.287901 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 17:08:03.287901 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 17:08:03.287901 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 17:08:03.287901 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 24 17:08:03.287901 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 24 17:08:03.287901 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [parameters: {'id': '64b2132cece44ee9a5a72ddd60f8a191', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 24 17:08:03.287901 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 24 17:08:03.287901 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:08:03.287901 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 17:08:03.287901 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:08:03.287901 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:03.287901 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:03.287901 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:03.287901 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:03.287901 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:03.290068 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:03.290068 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:03.290068 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:03.290068 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:03.290068 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:03.290068 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:03.290068 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 24 17:08:03.290068 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 24 17:08:03.290068 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:03.290068 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:03.290068 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 24 17:08:03.290068 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 24 17:08:03.290068 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 24 17:08:03.290068 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 24 17:08:03.290068 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 24 17:08:03.290068 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:08:03.296818 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 237/478] 10.222.0.18 () {60 vars in 1131 bytes} [Wed Aug 24 17:08:03 2022] POST /identity/v3/roles => generated 144 bytes in 255 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:03.296960 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b60a613-ed85-4030-8de9-f09cf3b9eeb2 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:03.300574 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:03.463300 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b60a613-ed85-4030-8de9-f09cf3b9eeb2 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:03.514583 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb89f280-e122-4dcb-a63d-088a135d4ce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:03.515251 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb89f280-e122-4dcb-a63d-088a135d4ce1 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:03.515531 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb89f280-e122-4dcb-a63d-088a135d4ce1 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:03.515803 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb89f280-e122-4dcb-a63d-088a135d4ce1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:03.532271 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb89f280-e122-4dcb-a63d-088a135d4ce1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:08:03.536934 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-eb89f280-e122-4dcb-a63d-088a135d4ce1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 17:08:03.536934 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:03.536934 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:03.536934 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:03.536934 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:03.536934 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:03.536934 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:03.536934 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:03.536934 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:03.536934 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:03.536934 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:03.536934 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:03.536934 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:08:03.536934 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:08:03.536934 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:08:03.536934 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:08:03.537813 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:08:03.537813 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:08:03.537813 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:08:03.537813 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:08:03.537813 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:03.537813 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:03.537813 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:03.537813 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:03.537813 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:03.537813 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:03.537813 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:03.537813 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:03.537813 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:03.537813 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:03.537813 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:03.537813 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:03.538597 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:03.538597 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:03.538597 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:03.538597 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:03.538597 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:08:03.538597 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:08:03.538597 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:08:03.538597 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:08:03.538597 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:08:03.538597 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:08:03.538597 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 17:08:03.538597 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:08:03.538597 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 242/479] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Aug 24 17:08:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:03.543828 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bff0b5-777d-449c-a90d-bb16a07ffe77 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:03.673086 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bff0b5-777d-449c-a90d-bb16a07ffe77 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:03.717589 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e16b9e-6f0d-4c4e-90e7-98b40bd71bbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:03.718209 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e16b9e-6f0d-4c4e-90e7-98b40bd71bbb None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:03.718491 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e16b9e-6f0d-4c4e-90e7-98b40bd71bbb None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:03.718765 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e16b9e-6f0d-4c4e-90e7-98b40bd71bbb None admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:03.719467 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e16b9e-6f0d-4c4e-90e7-98b40bd71bbb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:03.719776 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e16b9e-6f0d-4c4e-90e7-98b40bd71bbb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:03.721166 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e16b9e-6f0d-4c4e-90e7-98b40bd71bbb None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:03.729952 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 238/480] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 24 17:08:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:04.052716 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fec5ee-7de3-44e8-a5fd-5aacf6b57ae5 None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:04.053231 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fec5ee-7de3-44e8-a5fd-5aacf6b57ae5 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:04.053448 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fec5ee-7de3-44e8-a5fd-5aacf6b57ae5 None None] PATH_INFO: `/` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:04.055539 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 243/481] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:08:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:08:04.061179 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d2b2e1-8292-4eb8-8568-4d892d7df21f None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:04.061473 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d2b2e1-8292-4eb8-8568-4d892d7df21f None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:04.061754 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d2b2e1-8292-4eb8-8568-4d892d7df21f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:04.081955 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-11d2b2e1-8292-4eb8-8568-4d892d7df21f None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:08:04.083178 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-11d2b2e1-8292-4eb8-8568-4d892d7df21f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:08:04.144276 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:04.169859 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 239/482] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:08:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:08:04.500408 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fec5ee-7de3-44e8-a5fd-5aacf6b57ae5 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:04.599975 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:04.650590 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fec5ee-7de3-44e8-a5fd-5aacf6b57ae5 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:04.695594 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c49d4a6-5779-4c89-a92a-7380fcced7d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:04.696228 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c49d4a6-5779-4c89-a92a-7380fcced7d4 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:04.696508 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c49d4a6-5779-4c89-a92a-7380fcced7d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:04.696779 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c49d4a6-5779-4c89-a92a-7380fcced7d4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:04.717785 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c49d4a6-5779-4c89-a92a-7380fcced7d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:08:04.722534 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-2c49d4a6-5779-4c89-a92a-7380fcced7d4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:08:04.722534 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:04.722534 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:04.722534 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:04.722534 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:04.722534 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:04.722534 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:04.722534 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:04.722534 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:04.722534 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:04.722534 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:04.722534 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:04.722534 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:08:04.722534 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:08:04.722534 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:08:04.722534 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:08:04.723470 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:08:04.723470 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:08:04.723470 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:08:04.723470 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:08:04.723470 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:04.723470 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:04.723470 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:04.723470 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:04.723470 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:04.723470 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:04.723470 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:04.723470 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:04.723470 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:04.723470 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:04.723470 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:04.723470 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:04.724530 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:04.724530 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:04.724530 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:04.724530 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:04.724530 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:08:04.724530 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:08:04.724530 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:08:04.724530 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:08:04.724530 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:08:04.724530 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:08:04.724530 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:08:04.724530 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:08:04.724530 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 244/483] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Aug 24 17:08:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:04.731161 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d2b2e1-8292-4eb8-8568-4d892d7df21f None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:04.740519 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9780471b-a6da-4356-ba45-a8770111f8e5 None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:04.740805 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9780471b-a6da-4356-ba45-a8770111f8e5 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:04.741079 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9780471b-a6da-4356-ba45-a8770111f8e5 None None] PATH_INFO: `/` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:04.742673 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 245/484] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:08:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:08:04.748248 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb50d63-24c6-4428-8bc6-0e01a440f165 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:04.748532 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb50d63-24c6-4428-8bc6-0e01a440f165 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:04.748812 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb50d63-24c6-4428-8bc6-0e01a440f165 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:04.769631 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-dbb50d63-24c6-4428-8bc6-0e01a440f165 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:08:04.770754 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-dbb50d63-24c6-4428-8bc6-0e01a440f165 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:08:04.881163 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 246/485] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:08:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 17:08:04.904645 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d2b2e1-8292-4eb8-8568-4d892d7df21f None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:04.945659 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e02d51-a6a3-4236-b3ae-eb3c6578f866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:04.946279 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e02d51-a6a3-4236-b3ae-eb3c6578f866 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:04.946562 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e02d51-a6a3-4236-b3ae-eb3c6578f866 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:04.946839 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e02d51-a6a3-4236-b3ae-eb3c6578f866 None admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:04.947543 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e02d51-a6a3-4236-b3ae-eb3c6578f866 None admin] RBAC: Adding query filter params (name=service) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:04.948703 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e02d51-a6a3-4236-b3ae-eb3c6578f866 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:04.950035 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e02d51-a6a3-4236-b3ae-eb3c6578f866 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:04.959077 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 240/486] 10.222.0.18 () {56 vars in 1103 bytes} [Wed Aug 24 17:08:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:04.964467 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb50d63-24c6-4428-8bc6-0e01a440f165 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:05.021297 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:05.073094 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb50d63-24c6-4428-8bc6-0e01a440f165 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:05.078169 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b158f160-da29-4ec1-8ce8-b59a8e9ef2d5 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:05.078454 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b158f160-da29-4ec1-8ce8-b59a8e9ef2d5 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:05.078729 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b158f160-da29-4ec1-8ce8-b59a8e9ef2d5 None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:05.080245 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 241/487] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:08:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:08:05.083326 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e89844-3444-4211-8e1e-dc89fe4114e4 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:05.083668 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e89844-3444-4211-8e1e-dc89fe4114e4 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:05.083946 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e89844-3444-4211-8e1e-dc89fe4114e4 None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:05.085490 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 242/488] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:08:05 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 17:08:05.086800 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc057ae5-b8bc-4d32-b42a-bc92cdd258df None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:05.087299 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc057ae5-b8bc-4d32-b42a-bc92cdd258df None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:05.087791 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc057ae5-b8bc-4d32-b42a-bc92cdd258df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:05.109381 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-cc057ae5-b8bc-4d32-b42a-bc92cdd258df None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:08:05.110567 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-cc057ae5-b8bc-4d32-b42a-bc92cdd258df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:08:05.116923 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804787c1-6e06-4bb1-91e6-663968d5de8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:05.117565 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804787c1-6e06-4bb1-91e6-663968d5de8b None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:05.117844 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804787c1-6e06-4bb1-91e6-663968d5de8b None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:05.118116 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804787c1-6e06-4bb1-91e6-663968d5de8b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:05.129296 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804787c1-6e06-4bb1-91e6-663968d5de8b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:05.130866 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804787c1-6e06-4bb1-91e6-663968d5de8b None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:05.146152 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-804787c1-6e06-4bb1-91e6-663968d5de8b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:08:05.146152 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:05.146152 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:05.146152 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:05.146152 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:05.146152 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:05.146152 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:05.146152 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:05.146152 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:05.146152 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:05.146152 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:05.146152 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:05.146152 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:08:05.146152 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:08:05.146152 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:08:05.146152 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:08:05.147121 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:05.147121 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:05.147121 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:05.147121 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:05.147121 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:05.147121 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:05.147121 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:05.147121 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:05.147121 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:05.147121 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:05.147121 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:05.147121 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:05.147121 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:05.147121 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:05.147121 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:05.147121 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:05.147121 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:08:05.148256 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:08:05.148256 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:08:05.148256 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:08:05.148256 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 247/489] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 24 17:08:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:05.148530 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63155ff3-144e-43bc-905d-514db6fe66c8 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:05.149037 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63155ff3-144e-43bc-905d-514db6fe66c8 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:05.149037 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63155ff3-144e-43bc-905d-514db6fe66c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:05.158094 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:05.167685 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-63155ff3-144e-43bc-905d-514db6fe66c8 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:08:05.169493 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-63155ff3-144e-43bc-905d-514db6fe66c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:08:05.186336 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 243/490] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:08:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 17:08:05.188446 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc057ae5-b8bc-4d32-b42a-bc92cdd258df None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:05.240545 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 248/491] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:08:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:08:05.241407 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63155ff3-144e-43bc-905d-514db6fe66c8 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:05.281533 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:05.360274 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63155ff3-144e-43bc-905d-514db6fe66c8 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:05.380823 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc057ae5-b8bc-4d32-b42a-bc92cdd258df None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:05.402926 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5904baf0-ea16-4092-9b24-b99a0abcfd13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:05.402926 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5904baf0-ea16-4092-9b24-b99a0abcfd13 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:05.402926 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5904baf0-ea16-4092-9b24-b99a0abcfd13 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:05.402926 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5904baf0-ea16-4092-9b24-b99a0abcfd13 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:05.411922 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5904baf0-ea16-4092-9b24-b99a0abcfd13 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:08:05.417067 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-5904baf0-ea16-4092-9b24-b99a0abcfd13 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:08:05.417067 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:05.417067 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:05.417067 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:05.417067 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:05.417067 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:05.417067 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:05.417067 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:05.417067 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:05.417067 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:05.417067 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:05.417067 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:05.417067 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:08:05.417067 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:08:05.417067 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:08:05.417067 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:08:05.417949 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:08:05.417949 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:08:05.417949 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:08:05.417949 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:08:05.417949 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:05.417949 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:05.417949 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:05.417949 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:05.417949 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:05.417949 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:05.417949 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:05.417949 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:05.417949 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:05.417949 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:05.417949 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:05.417949 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:05.418956 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:05.418956 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:05.418956 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:05.418956 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:05.418956 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:08:05.418956 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:08:05.418956 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:08:05.418956 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:08:05.418956 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:08:05.418956 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:08:05.418956 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:08:05.418956 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:08:05.418956 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 249/492] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Aug 24 17:08:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:05.424095 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5904baf0-ea16-4092-9b24-b99a0abcfd13 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:05.476326 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d95b78-90c6-4ddb-8e57-5b590fa30f51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:05.476993 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d95b78-90c6-4ddb-8e57-5b590fa30f51 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:05.477225 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d95b78-90c6-4ddb-8e57-5b590fa30f51 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:05.477494 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d95b78-90c6-4ddb-8e57-5b590fa30f51 None admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:05.478154 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d95b78-90c6-4ddb-8e57-5b590fa30f51 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:05.478454 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d95b78-90c6-4ddb-8e57-5b590fa30f51 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:05.479870 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d95b78-90c6-4ddb-8e57-5b590fa30f51 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:05.492128 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 244/493] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 24 17:08:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:05.492733 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d95b78-90c6-4ddb-8e57-5b590fa30f51 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:05.530241 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:05.546048 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:05.552875 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5904baf0-ea16-4092-9b24-b99a0abcfd13 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:05.595363 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0432c0f-3a59-4632-913f-ce508a1490ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:05.595984 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0432c0f-3a59-4632-913f-ce508a1490ef None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:05.596267 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0432c0f-3a59-4632-913f-ce508a1490ef None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:05.596544 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0432c0f-3a59-4632-913f-ce508a1490ef None admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:05.597210 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0432c0f-3a59-4632-913f-ce508a1490ef None admin] RBAC: Adding query filter params (name=service) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:05.597517 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0432c0f-3a59-4632-913f-ce508a1490ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:05.598890 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0432c0f-3a59-4632-913f-ce508a1490ef None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:05.605775 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d95b78-90c6-4ddb-8e57-5b590fa30f51 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:05.609957 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 250/494] 10.222.0.18 () {56 vars in 1103 bytes} [Wed Aug 24 17:08:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:05.611030 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0432c0f-3a59-4632-913f-ce508a1490ef None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:05.656287 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcc9c5d-e76e-43c0-953f-f975a08aa528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:05.656914 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcc9c5d-e76e-43c0-953f-f975a08aa528 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:05.657194 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcc9c5d-e76e-43c0-953f-f975a08aa528 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:05.657467 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcc9c5d-e76e-43c0-953f-f975a08aa528 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:05.669531 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcc9c5d-e76e-43c0-953f-f975a08aa528 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:08:05.677596 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-bbcc9c5d-e76e-43c0-953f-f975a08aa528 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:08:05.677596 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:05.677596 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:05.677596 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:05.677596 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:05.677596 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:05.677596 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:05.677596 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:05.677596 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:05.677596 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:05.677596 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:05.677596 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:05.677596 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:08:05.677596 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:08:05.677596 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:08:05.677596 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:08:05.678456 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:08:05.678456 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:08:05.678456 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:08:05.678456 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:08:05.678456 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:05.678456 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:05.678456 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:05.678456 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:05.678456 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:05.678456 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:05.678456 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:05.678456 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:05.678456 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:05.678456 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:05.678456 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:05.678456 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:05.679319 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:05.679319 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:05.679319 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:05.679319 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:05.679319 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:08:05.679319 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:08:05.679319 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:08:05.679319 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:08:05.679319 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:08:05.679319 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:08:05.679319 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:08:05.679319 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:08:05.679923 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 245/495] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Aug 24 17:08:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:05.680601 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcc9c5d-e76e-43c0-953f-f975a08aa528 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:05.728004 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0432c0f-3a59-4632-913f-ce508a1490ef None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:05.765033 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efdb192-00f4-41ee-b748-37191db8ccfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:05.765667 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efdb192-00f4-41ee-b748-37191db8ccfe None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:05.765945 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efdb192-00f4-41ee-b748-37191db8ccfe None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:05.766484 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efdb192-00f4-41ee-b748-37191db8ccfe None admin] PATH_INFO: `/v3/users/nova` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:05.774117 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efdb192-00f4-41ee-b748-37191db8ccfe None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:05.775624 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efdb192-00f4-41ee-b748-37191db8ccfe None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:05.786574 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:05.787992 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-3efdb192-00f4-41ee-b748-37191db8ccfe None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 17:08:05.787992 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:05.787992 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:05.787992 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:05.787992 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:05.787992 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:05.787992 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:05.787992 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:05.787992 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:05.787992 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:05.787992 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:05.787992 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:05.787992 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:08:05.787992 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:08:05.787992 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:08:05.787992 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:08:05.788879 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:05.788879 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:05.788879 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:08:05.788879 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:08:05.788879 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:08:05.788879 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:08:05.788879 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:05.788879 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:05.788879 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:05.788879 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:05.788879 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:05.788879 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:05.788879 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:05.788879 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:05.788879 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:05.788879 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:05.788879 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:05.788879 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:05.789786 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:05.789786 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:05.789786 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:08:05.789786 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:08:05.789786 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:08:05.789786 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:08:05.789786 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:08:05.789786 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:08:05.789786 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 17:08:05.789786 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:08:05.789786 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 251/496] 10.222.0.18 () {56 vars in 1093 bytes} [Wed Aug 24 17:08:05 2022] GET /identity/v3/users/nova => generated 82 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:05.790335 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efdb192-00f4-41ee-b748-37191db8ccfe None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:05.810156 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcc9c5d-e76e-43c0-953f-f975a08aa528 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:05.811024 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:05.832494 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93882b09-146a-42b3-981f-d7998354333a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:05.833117 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93882b09-146a-42b3-981f-d7998354333a None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:05.833491 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93882b09-146a-42b3-981f-d7998354333a None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:05.833766 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93882b09-146a-42b3-981f-d7998354333a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:05.843964 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93882b09-146a-42b3-981f-d7998354333a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:08:05.848743 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-93882b09-146a-42b3-981f-d7998354333a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:08:05.848743 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:05.848743 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:05.848743 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:05.848743 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:05.848743 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:05.848743 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:05.848743 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:05.848743 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:05.848743 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:05.848743 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:05.848743 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:05.848743 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:08:05.848743 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:08:05.848743 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:08:05.848743 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:08:05.849688 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:08:05.849688 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:08:05.849688 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:08:05.849688 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:08:05.849688 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:05.849688 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:05.849688 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:05.849688 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:05.849688 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:05.849688 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:05.849688 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:05.849688 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:05.849688 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:05.849688 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:05.849688 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:05.849688 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:05.850494 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:05.850494 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:05.850494 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:05.850494 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:05.850494 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:08:05.850494 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:08:05.850494 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:08:05.850494 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:08:05.850494 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:08:05.850494 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:08:05.850494 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:08:05.850494 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:08:05.850494 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 246/497] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Aug 24 17:08:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:05.851881 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93882b09-146a-42b3-981f-d7998354333a None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:05.903836 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efdb192-00f4-41ee-b748-37191db8ccfe None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:05.938860 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5e0780-08f0-49f5-af19-3dd61067c9b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:05.939543 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5e0780-08f0-49f5-af19-3dd61067c9b1 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:05.939818 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5e0780-08f0-49f5-af19-3dd61067c9b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:05.940082 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5e0780-08f0-49f5-af19-3dd61067c9b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:05.951567 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5e0780-08f0-49f5-af19-3dd61067c9b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:05.953021 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5e0780-08f0-49f5-af19-3dd61067c9b1 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:05.955901 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93882b09-146a-42b3-981f-d7998354333a None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:05.967293 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-4a5e0780-08f0-49f5-af19-3dd61067c9b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:08:05.967293 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:05.967293 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:05.967293 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:05.967293 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:05.967293 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:05.967293 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:05.967293 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:05.967293 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:05.967293 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:05.967293 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:05.967293 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:05.967293 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:08:05.967293 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:08:05.967293 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:08:05.967293 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:08:05.968396 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:05.968396 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:05.968396 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:05.968396 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:05.968396 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:05.968396 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:05.968396 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:05.968396 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:05.968396 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:05.968396 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:05.968396 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:05.968396 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:05.968396 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:05.968396 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:05.968396 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:05.968396 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:05.968396 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:08:05.969237 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:08:05.969237 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:08:05.969237 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:08:05.969237 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 252/498] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 24 17:08:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:05.969449 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5e0780-08f0-49f5-af19-3dd61067c9b1 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:05.996749 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b779d9-8e38-4e19-904d-bb76a479d073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:05.998029 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b779d9-8e38-4e19-904d-bb76a479d073 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:05.998397 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b779d9-8e38-4e19-904d-bb76a479d073 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:05.998784 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b779d9-8e38-4e19-904d-bb76a479d073 None admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:05.999643 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b779d9-8e38-4e19-904d-bb76a479d073 None admin] RBAC: Adding query filter params (name=service) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:05.999941 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b779d9-8e38-4e19-904d-bb76a479d073 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:06.001637 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b779d9-8e38-4e19-904d-bb76a479d073 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:06.014706 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 247/499] 10.222.0.18 () {56 vars in 1103 bytes} [Wed Aug 24 17:08:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:06.014706 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b779d9-8e38-4e19-904d-bb76a479d073 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:06.030723 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:06.051546 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:06.086345 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5e0780-08f0-49f5-af19-3dd61067c9b1 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:06.111314 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b779d9-8e38-4e19-904d-bb76a479d073 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:06.122673 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b046802c-376d-4dbb-a3cf-876cc724e5cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:06.123311 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b046802c-376d-4dbb-a3cf-876cc724e5cc None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:06.123591 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b046802c-376d-4dbb-a3cf-876cc724e5cc None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:06.123895 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b046802c-376d-4dbb-a3cf-876cc724e5cc None admin] PATH_INFO: `/v3/users` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:06.124693 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b046802c-376d-4dbb-a3cf-876cc724e5cc None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:06.125122 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b046802c-376d-4dbb-a3cf-876cc724e5cc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:06.126813 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b046802c-376d-4dbb-a3cf-876cc724e5cc None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:06.145796 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae6d6a8-e5b6-46f2-966e-92c4a113bc5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:06.146448 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae6d6a8-e5b6-46f2-966e-92c4a113bc5d None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:06.146730 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae6d6a8-e5b6-46f2-966e-92c4a113bc5d None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:06.147025 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae6d6a8-e5b6-46f2-966e-92c4a113bc5d None admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:06.147685 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae6d6a8-e5b6-46f2-966e-92c4a113bc5d None admin] RBAC: Adding query filter params (name=service) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:06.147983 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae6d6a8-e5b6-46f2-966e-92c4a113bc5d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:06.149455 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae6d6a8-e5b6-46f2-966e-92c4a113bc5d None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:06.161513 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 248/500] 10.222.0.18 () {56 vars in 1103 bytes} [Wed Aug 24 17:08:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:06.161513 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae6d6a8-e5b6-46f2-966e-92c4a113bc5d None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:06.171066 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 253/501] 10.222.0.18 () {56 vars in 1133 bytes} [Wed Aug 24 17:08:05 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:06.171066 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b046802c-376d-4dbb-a3cf-876cc724e5cc None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:06.245898 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:06.267483 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae6d6a8-e5b6-46f2-966e-92c4a113bc5d None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:06.271962 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b046802c-376d-4dbb-a3cf-876cc724e5cc None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:06.298943 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:06.305571 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b2834d-9caa-46a1-a463-a7185faa8203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:06.306201 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b2834d-9caa-46a1-a463-a7185faa8203 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:06.306458 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b2834d-9caa-46a1-a463-a7185faa8203 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:06.309161 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b2834d-9caa-46a1-a463-a7185faa8203 None admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:06.309253 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aafd845-4246-43cf-8aa8-f8f50f329951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:06.309367 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b2834d-9caa-46a1-a463-a7185faa8203 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:06.309483 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aafd845-4246-43cf-8aa8-f8f50f329951 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:06.310086 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b2834d-9caa-46a1-a463-a7185faa8203 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:06.310167 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aafd845-4246-43cf-8aa8-f8f50f329951 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:06.310167 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aafd845-4246-43cf-8aa8-f8f50f329951 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:06.310284 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b2834d-9caa-46a1-a463-a7185faa8203 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:06.318998 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aafd845-4246-43cf-8aa8-f8f50f329951 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:06.320443 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aafd845-4246-43cf-8aa8-f8f50f329951 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:06.322562 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 249/502] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 24 17:08:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:06.323356 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b2834d-9caa-46a1-a463-a7185faa8203 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:06.336062 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-5aafd845-4246-43cf-8aa8-f8f50f329951 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:08:06.336062 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:06.336062 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:06.336062 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:06.336062 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:06.336062 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:06.336062 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:06.336062 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:06.336062 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:06.336062 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:06.336062 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:06.336062 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:06.336062 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:08:06.336062 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:08:06.336062 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:08:06.336062 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:08:06.338649 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:06.338649 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:06.338649 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:06.338649 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:06.338649 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:06.338649 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:06.338649 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:06.338649 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:06.338649 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:06.338649 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:06.338649 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:06.338649 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:06.338649 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:06.338649 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:06.338649 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:06.338649 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:06.338649 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:08:06.339738 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:08:06.339738 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:08:06.339738 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:08:06.339738 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 254/503] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 24 17:08:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:06.339738 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aafd845-4246-43cf-8aa8-f8f50f329951 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:06.428212 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b2834d-9caa-46a1-a463-a7185faa8203 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:06.434039 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aafd845-4246-43cf-8aa8-f8f50f329951 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:06.462342 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15b7e59-12b7-415f-a01b-69f7ed627d33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:06.463002 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15b7e59-12b7-415f-a01b-69f7ed627d33 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:06.463291 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15b7e59-12b7-415f-a01b-69f7ed627d33 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:06.463568 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15b7e59-12b7-415f-a01b-69f7ed627d33 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:06.467199 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95343c9-3508-4727-807a-b6b43a5249b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:06.467794 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95343c9-3508-4727-807a-b6b43a5249b0 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:06.468073 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95343c9-3508-4727-807a-b6b43a5249b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:06.468343 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95343c9-3508-4727-807a-b6b43a5249b0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:06.472785 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:06.473792 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15b7e59-12b7-415f-a01b-69f7ed627d33 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:06.475349 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15b7e59-12b7-415f-a01b-69f7ed627d33 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:06.490002 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-b15b7e59-12b7-415f-a01b-69f7ed627d33 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:08:06.490002 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:06.490002 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:06.490002 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:06.490002 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:06.490002 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:06.490002 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:06.490002 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:06.490002 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:06.490002 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:06.490002 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:06.490002 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:06.490002 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:08:06.490002 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:08:06.490002 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:08:06.490002 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:08:06.490893 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:06.490893 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:06.490893 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:06.490893 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:06.490893 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:06.490893 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:06.490893 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:06.490893 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:06.490893 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:06.490893 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:06.490893 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:06.490893 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:06.490893 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:06.490893 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:06.490893 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:06.490893 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:06.490893 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:08:06.491803 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:08:06.491803 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:08:06.491803 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:08:06.491803 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 250/504] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 24 17:08:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:06.492442 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0d84df-1442-4925-8b7e-fc91b6cf15fc None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:06.492723 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0d84df-1442-4925-8b7e-fc91b6cf15fc None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:06.492996 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0d84df-1442-4925-8b7e-fc91b6cf15fc None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:06.494346 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 251/505] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:08:06 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:08:06.494943 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0d84df-1442-4925-8b7e-fc91b6cf15fc None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:06.522768 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:06.542377 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95343c9-3508-4727-807a-b6b43a5249b0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:06.544598 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95343c9-3508-4727-807a-b6b43a5249b0 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:06.612649 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0d84df-1442-4925-8b7e-fc91b6cf15fc None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:06.645477 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5080835d-4beb-4026-a484-7e056c15e160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:06.646104 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5080835d-4beb-4026-a484-7e056c15e160 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:06.646383 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5080835d-4beb-4026-a484-7e056c15e160 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:06.646655 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5080835d-4beb-4026-a484-7e056c15e160 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:06.648485 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 255/506] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 24 17:08:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 17:08:06.648485 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95343c9-3508-4727-807a-b6b43a5249b0 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:06.655494 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5080835d-4beb-4026-a484-7e056c15e160 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:06.656936 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5080835d-4beb-4026-a484-7e056c15e160 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:06.669631 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-5080835d-4beb-4026-a484-7e056c15e160 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 24 17:08:06.669631 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:06.669631 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:06.669631 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:06.669631 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:06.669631 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:06.669631 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:06.669631 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:06.669631 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:06.669631 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:06.669631 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:06.669631 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:06.669631 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:08:06.669631 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:08:06.669631 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:08:06.669631 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:08:06.670543 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:06.670543 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:06.670543 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:08:06.670543 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:08:06.670543 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:08:06.670543 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:08:06.670543 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:06.670543 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:06.670543 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:06.670543 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:06.670543 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:06.670543 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:06.670543 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:06.670543 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:06.670543 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:06.670543 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:06.670543 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:06.670543 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:06.672080 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:06.672080 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:06.672080 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:08:06.672080 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:08:06.672080 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:08:06.672080 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:08:06.672080 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:08:06.672080 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:08:06.672080 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 24 17:08:06.672080 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:08:06.672080 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 252/507] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 24 17:08:06 2022] GET /identity/v3/users/glance => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:06.672080 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5080835d-4beb-4026-a484-7e056c15e160 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:06.723468 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:06.729541 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:06.747814 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95343c9-3508-4727-807a-b6b43a5249b0 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:06.769492 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5080835d-4beb-4026-a484-7e056c15e160 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:06.781177 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95bba32-2681-43eb-87f1-b1655a5acb13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:06.781776 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95bba32-2681-43eb-87f1-b1655a5acb13 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:06.782054 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95bba32-2681-43eb-87f1-b1655a5acb13 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:06.782327 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95bba32-2681-43eb-87f1-b1655a5acb13 None admin] PATH_INFO: `/v3/domains` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:06.782986 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95bba32-2681-43eb-87f1-b1655a5acb13 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:06.783296 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95bba32-2681-43eb-87f1-b1655a5acb13 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:06.784653 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95bba32-2681-43eb-87f1-b1655a5acb13 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:06.796270 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 256/508] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 24 17:08:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:06.797406 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15189cf3-8e62-4337-8f55-aeae05e52722 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:06.797693 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15189cf3-8e62-4337-8f55-aeae05e52722 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:06.797962 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15189cf3-8e62-4337-8f55-aeae05e52722 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:06.807746 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bdf6a3-d9aa-4cf1-bafe-57679e846d1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:06.808378 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bdf6a3-d9aa-4cf1-bafe-57679e846d1b None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:06.808655 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bdf6a3-d9aa-4cf1-bafe-57679e846d1b None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:06.808926 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bdf6a3-d9aa-4cf1-bafe-57679e846d1b None admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:06.809622 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bdf6a3-d9aa-4cf1-bafe-57679e846d1b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:06.809957 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bdf6a3-d9aa-4cf1-bafe-57679e846d1b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:06.811378 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bdf6a3-d9aa-4cf1-bafe-57679e846d1b None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:06.814717 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-15189cf3-8e62-4337-8f55-aeae05e52722 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:08:06.815914 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-15189cf3-8e62-4337-8f55-aeae05e52722 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:08:06.823130 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 253/509] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 24 17:08:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:06.823885 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bdf6a3-d9aa-4cf1-bafe-57679e846d1b None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:06.882838 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 257/510] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:08:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:08:06.883629 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15189cf3-8e62-4337-8f55-aeae05e52722 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:06.929506 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bdf6a3-d9aa-4cf1-bafe-57679e846d1b None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:06.943040 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:06.956882 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:06.965633 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddb0680-77e7-4b7b-a69c-d17a2be44bf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:06.966927 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddb0680-77e7-4b7b-a69c-d17a2be44bf1 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:06.967209 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddb0680-77e7-4b7b-a69c-d17a2be44bf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:06.967480 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddb0680-77e7-4b7b-a69c-d17a2be44bf1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:06.978394 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddb0680-77e7-4b7b-a69c-d17a2be44bf1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:06.980452 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddb0680-77e7-4b7b-a69c-d17a2be44bf1 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:06.990191 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15189cf3-8e62-4337-8f55-aeae05e52722 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:06.994835 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-dddb0680-77e7-4b7b-a69c-d17a2be44bf1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:08:06.994835 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:06.994835 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:06.994835 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:06.994835 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:06.994835 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:06.994835 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:06.994835 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:06.994835 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:06.994835 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:06.994835 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:06.994835 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:06.994835 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:08:06.994835 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:08:06.994835 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:08:06.994835 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:08:06.995908 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:06.995908 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:06.995908 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:06.995908 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:06.995908 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:06.995908 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:06.995908 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:06.995908 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:06.995908 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:06.995908 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:06.995908 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:06.995908 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:06.995908 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:06.995908 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:06.995908 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:06.995908 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:06.995908 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:08:06.996998 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:08:06.996998 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:08:06.996998 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:08:07.001014 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 254/511] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 24 17:08:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:07.001855 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddb0680-77e7-4b7b-a69c-d17a2be44bf1 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:07.030276 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a598a8-0ee8-4f6d-ab7a-4d028bdd0857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:07.031125 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a598a8-0ee8-4f6d-ab7a-4d028bdd0857 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:07.031561 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a598a8-0ee8-4f6d-ab7a-4d028bdd0857 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:07.031834 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a598a8-0ee8-4f6d-ab7a-4d028bdd0857 None admin] PATH_INFO: `/v3/users` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:07.032657 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a598a8-0ee8-4f6d-ab7a-4d028bdd0857 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:07.032965 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a598a8-0ee8-4f6d-ab7a-4d028bdd0857 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:07.034401 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a598a8-0ee8-4f6d-ab7a-4d028bdd0857 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:07.052028 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 258/512] 10.222.0.18 () {56 vars in 1137 bytes} [Wed Aug 24 17:08:06 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:07.052843 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a598a8-0ee8-4f6d-ab7a-4d028bdd0857 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:07.107847 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddb0680-77e7-4b7b-a69c-d17a2be44bf1 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:07.141782 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec056c01-11fd-46e0-8fab-c04b31b9313c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:07.142335 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec056c01-11fd-46e0-8fab-c04b31b9313c None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:07.142618 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec056c01-11fd-46e0-8fab-c04b31b9313c None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:07.142885 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec056c01-11fd-46e0-8fab-c04b31b9313c None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:07.150287 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec056c01-11fd-46e0-8fab-c04b31b9313c None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:07.151766 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec056c01-11fd-46e0-8fab-c04b31b9313c None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:07.155390 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a598a8-0ee8-4f6d-ab7a-4d028bdd0857 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:07.163375 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-ec056c01-11fd-46e0-8fab-c04b31b9313c None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 17:08:07.163375 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:07.163375 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:07.163375 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:07.163375 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:07.163375 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:07.163375 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:07.163375 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:07.163375 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:07.163375 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:07.163375 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:07.163375 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:07.163375 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:08:07.163375 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:08:07.163375 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:08:07.163375 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:08:07.164444 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:07.164444 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:07.164444 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:08:07.164444 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:08:07.164444 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:08:07.164444 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:08:07.164444 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:07.164444 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:07.164444 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:07.164444 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:07.164444 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:07.164444 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:07.164444 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:07.164444 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:07.164444 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:07.164444 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:07.164444 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:07.164444 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:07.165943 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:07.165943 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:07.165943 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:08:07.165943 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:08:07.165943 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:08:07.165943 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:08:07.165943 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:08:07.165943 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:08:07.165943 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 17:08:07.165943 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:08:07.165943 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 255/513] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Aug 24 17:08:07 2022] GET /identity/v3/users/neutron => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:07.165943 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec056c01-11fd-46e0-8fab-c04b31b9313c None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:07.183154 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:07.190610 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b19303-4cfd-4c57-b65e-81d9993ec600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:07.197714 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b19303-4cfd-4c57-b65e-81d9993ec600 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:07.197714 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b19303-4cfd-4c57-b65e-81d9993ec600 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:07.197714 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b19303-4cfd-4c57-b65e-81d9993ec600 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:07.198022 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:07.199406 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b19303-4cfd-4c57-b65e-81d9993ec600 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:07.201333 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b19303-4cfd-4c57-b65e-81d9993ec600 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:07.212577 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-99b19303-4cfd-4c57-b65e-81d9993ec600 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 17:08:07.212577 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:07.212577 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:07.212577 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:07.212577 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:07.212577 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:07.212577 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:07.212577 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:07.212577 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:07.212577 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:07.212577 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:07.212577 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:07.212577 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:08:07.212577 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:08:07.212577 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:08:07.212577 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:08:07.213629 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:07.213629 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:07.213629 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:08:07.213629 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:08:07.213629 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:08:07.213629 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:08:07.213629 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:07.213629 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:07.213629 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:07.213629 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:07.213629 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:07.213629 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:07.213629 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:07.213629 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:07.213629 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:07.213629 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:07.213629 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:07.213629 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:07.214721 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:07.214721 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:07.214721 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:08:07.214721 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:08:07.214721 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:08:07.214721 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:08:07.214721 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:08:07.214721 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:08:07.214721 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 17:08:07.214721 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:08:07.218837 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 259/514] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 24 17:08:07 2022] GET /identity/v3/users/cinder => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:07.219632 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b19303-4cfd-4c57-b65e-81d9993ec600 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:07.272645 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec056c01-11fd-46e0-8fab-c04b31b9313c None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:07.305846 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5519ddf3-f95f-4dd1-87fd-d6a88dbf583f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:07.306450 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5519ddf3-f95f-4dd1-87fd-d6a88dbf583f None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:07.306726 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5519ddf3-f95f-4dd1-87fd-d6a88dbf583f None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:07.307027 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5519ddf3-f95f-4dd1-87fd-d6a88dbf583f None admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:07.307667 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5519ddf3-f95f-4dd1-87fd-d6a88dbf583f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:07.307963 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5519ddf3-f95f-4dd1-87fd-d6a88dbf583f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:07.309333 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5519ddf3-f95f-4dd1-87fd-d6a88dbf583f None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:07.316191 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b19303-4cfd-4c57-b65e-81d9993ec600 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:07.321860 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 256/515] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 24 17:08:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:07.322650 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5519ddf3-f95f-4dd1-87fd-d6a88dbf583f None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:07.349634 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ee4fdc-9150-42d4-bd46-f94c984fde7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:07.350235 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ee4fdc-9150-42d4-bd46-f94c984fde7e None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:07.350515 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ee4fdc-9150-42d4-bd46-f94c984fde7e None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:07.350788 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ee4fdc-9150-42d4-bd46-f94c984fde7e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:07.409735 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:07.410251 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:07.422224 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ee4fdc-9150-42d4-bd46-f94c984fde7e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:07.423731 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ee4fdc-9150-42d4-bd46-f94c984fde7e None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:07.425227 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5519ddf3-f95f-4dd1-87fd-d6a88dbf583f None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:07.459672 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d930af-c50d-4070-8f56-0d24079adc6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:07.460290 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d930af-c50d-4070-8f56-0d24079adc6d None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:07.460566 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d930af-c50d-4070-8f56-0d24079adc6d None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:07.460832 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d930af-c50d-4070-8f56-0d24079adc6d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:07.470961 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d930af-c50d-4070-8f56-0d24079adc6d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:07.472428 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d930af-c50d-4070-8f56-0d24079adc6d None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:07.486526 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-e7d930af-c50d-4070-8f56-0d24079adc6d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:08:07.486526 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:07.486526 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:07.486526 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:07.486526 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:07.486526 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:07.486526 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:07.486526 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:07.486526 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:07.486526 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:07.486526 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:07.486526 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:07.486526 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:08:07.486526 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:08:07.486526 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:08:07.486526 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:08:07.487637 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:07.487637 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:07.487637 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:07.487637 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:07.487637 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:07.487637 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:07.487637 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:07.487637 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:07.487637 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:07.487637 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:07.487637 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:07.487637 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:07.487637 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:07.487637 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:07.487637 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:07.487637 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:07.487637 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:08:07.488662 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:08:07.488662 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:08:07.488662 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:08:07.488662 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 257/516] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 24 17:08:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:07.488921 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d930af-c50d-4070-8f56-0d24079adc6d None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:07.523955 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 260/517] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 24 17:08:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 17:08:07.524981 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ee4fdc-9150-42d4-bd46-f94c984fde7e None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:07.586615 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d930af-c50d-4070-8f56-0d24079adc6d None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:07.608374 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:07.619368 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f84c07-312a-4d13-974d-b99b09ead3f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:07.619968 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ee4fdc-9150-42d4-bd46-f94c984fde7e None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:07.620076 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f84c07-312a-4d13-974d-b99b09ead3f9 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:07.620248 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f84c07-312a-4d13-974d-b99b09ead3f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:07.620522 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f84c07-312a-4d13-974d-b99b09ead3f9 None admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:07.621247 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f84c07-312a-4d13-974d-b99b09ead3f9 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:07.621553 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f84c07-312a-4d13-974d-b99b09ead3f9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:07.623602 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f84c07-312a-4d13-974d-b99b09ead3f9 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:07.635058 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:07.654401 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095bdf62-5117-415f-8b87-52ab5b58dc56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:07.655038 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095bdf62-5117-415f-8b87-52ab5b58dc56 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:07.655320 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095bdf62-5117-415f-8b87-52ab5b58dc56 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:07.655593 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095bdf62-5117-415f-8b87-52ab5b58dc56 None admin] PATH_INFO: `/v3/users` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:07.656321 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095bdf62-5117-415f-8b87-52ab5b58dc56 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:07.658720 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095bdf62-5117-415f-8b87-52ab5b58dc56 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:07.660273 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095bdf62-5117-415f-8b87-52ab5b58dc56 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:07.663040 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 258/518] 10.222.0.18 () {56 vars in 1139 bytes} [Wed Aug 24 17:08:07 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:07.664184 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f84c07-312a-4d13-974d-b99b09ead3f9 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:07.683381 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 261/519] 10.222.0.18 () {56 vars in 1137 bytes} [Wed Aug 24 17:08:07 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:07.684322 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095bdf62-5117-415f-8b87-52ab5b58dc56 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:07.765572 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f84c07-312a-4d13-974d-b99b09ead3f9 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:07.783156 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095bdf62-5117-415f-8b87-52ab5b58dc56 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:07.799257 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017cf3f6-1e52-40dd-bcdf-606980430327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:07.799883 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017cf3f6-1e52-40dd-bcdf-606980430327 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:07.800161 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017cf3f6-1e52-40dd-bcdf-606980430327 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:07.800433 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017cf3f6-1e52-40dd-bcdf-606980430327 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:07.806966 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017cf3f6-1e52-40dd-bcdf-606980430327 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:07.808439 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017cf3f6-1e52-40dd-bcdf-606980430327 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:07.815455 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a984a1-a2cd-438a-83a2-82ac56e1ab65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:07.816058 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a984a1-a2cd-438a-83a2-82ac56e1ab65 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:07.816338 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a984a1-a2cd-438a-83a2-82ac56e1ab65 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:07.816613 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a984a1-a2cd-438a-83a2-82ac56e1ab65 None admin] PATH_INFO: `/v3/domains` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:07.817253 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a984a1-a2cd-438a-83a2-82ac56e1ab65 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:07.817558 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a984a1-a2cd-438a-83a2-82ac56e1ab65 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:07.818919 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a984a1-a2cd-438a-83a2-82ac56e1ab65 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:07.820736 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-017cf3f6-1e52-40dd-bcdf-606980430327 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:08:07.820736 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:07.820736 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:07.820736 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:07.820736 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:07.820736 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:07.820736 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:07.820736 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:07.820736 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:07.820736 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:07.820736 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:07.820736 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:07.820736 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:08:07.820736 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:08:07.820736 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:08:07.821727 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:08:07.821727 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:07.821727 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:07.821727 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:07.821727 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:07.821727 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:07.821727 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:07.821727 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:07.821727 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:07.821727 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:07.821727 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:07.821727 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:07.821727 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:07.821727 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:07.821727 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:07.821727 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:07.821727 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:07.822799 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:08:07.822799 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:08:07.822799 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:08:07.822799 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:08:07.822799 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:08:07.822799 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:08:07.822799 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:08:07.822799 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:08:07.822799 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 259/520] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 24 17:08:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:07.822799 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017cf3f6-1e52-40dd-bcdf-606980430327 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:07.837907 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 262/521] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 24 17:08:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:07.837907 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a984a1-a2cd-438a-83a2-82ac56e1ab65 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:07.843023 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:07.882851 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:07.927096 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017cf3f6-1e52-40dd-bcdf-606980430327 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:07.930977 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a984a1-a2cd-438a-83a2-82ac56e1ab65 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:07.960715 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbb5661-4fad-4acf-8b03-ce15b0fe1fec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:07.961433 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbb5661-4fad-4acf-8b03-ce15b0fe1fec None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:07.961714 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbb5661-4fad-4acf-8b03-ce15b0fe1fec None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:07.962271 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbb5661-4fad-4acf-8b03-ce15b0fe1fec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:07.965215 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cc6b58-89e2-4045-bbc0-f771b68b1581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:07.966426 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cc6b58-89e2-4045-bbc0-f771b68b1581 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:07.966706 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cc6b58-89e2-4045-bbc0-f771b68b1581 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:07.967010 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cc6b58-89e2-4045-bbc0-f771b68b1581 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:07.973611 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbb5661-4fad-4acf-8b03-ce15b0fe1fec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:07.975106 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbb5661-4fad-4acf-8b03-ce15b0fe1fec None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:07.988864 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-1cbb5661-4fad-4acf-8b03-ce15b0fe1fec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:08:07.988864 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:07.988864 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:07.988864 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:07.988864 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:07.988864 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:07.988864 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:07.988864 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:07.988864 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:07.988864 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:07.988864 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:07.988864 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:07.988864 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:08:07.988864 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:08:07.988864 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:08:07.988864 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:08:07.989892 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:07.989892 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:07.989892 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:07.989892 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:07.989892 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:07.989892 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:07.989892 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:07.989892 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:07.989892 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:07.989892 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:07.989892 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:07.989892 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:07.989892 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:07.989892 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:07.989892 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:07.989892 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:07.989892 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:08:07.990762 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:08:07.990762 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:08:07.990762 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:08:07.990762 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 260/522] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 24 17:08:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:07.990762 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbb5661-4fad-4acf-8b03-ce15b0fe1fec None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:08.040450 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cc6b58-89e2-4045-bbc0-f771b68b1581 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:08.041913 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cc6b58-89e2-4045-bbc0-f771b68b1581 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:08.049587 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:08.097626 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbb5661-4fad-4acf-8b03-ce15b0fe1fec None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:08.104230 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:08.133239 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cea7831-c658-4b66-8001-c63ea7701731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:08.133843 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cea7831-c658-4b66-8001-c63ea7701731 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:08.134121 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cea7831-c658-4b66-8001-c63ea7701731 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:08.134390 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cea7831-c658-4b66-8001-c63ea7701731 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:08.139528 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 263/523] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 24 17:08:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 17:08:08.140368 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cc6b58-89e2-4045-bbc0-f771b68b1581 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:08.200109 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cea7831-c658-4b66-8001-c63ea7701731 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:08.202066 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cea7831-c658-4b66-8001-c63ea7701731 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:08.235515 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cc6b58-89e2-4045-bbc0-f771b68b1581 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:08.242803 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:08.268867 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8a5e16-0442-44d5-889c-3d46d00c1e7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:08.269470 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8a5e16-0442-44d5-889c-3d46d00c1e7b None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:08.269741 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8a5e16-0442-44d5-889c-3d46d00c1e7b None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:08.270006 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8a5e16-0442-44d5-889c-3d46d00c1e7b None admin] PATH_INFO: `/v3/projects` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:08.270681 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8a5e16-0442-44d5-889c-3d46d00c1e7b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:08.271010 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8a5e16-0442-44d5-889c-3d46d00c1e7b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:08.272418 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8a5e16-0442-44d5-889c-3d46d00c1e7b None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:08.289548 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 264/524] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Aug 24 17:08:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:08.290329 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8a5e16-0442-44d5-889c-3d46d00c1e7b None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:08.299920 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 261/525] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 24 17:08:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 17:08:08.300641 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cea7831-c658-4b66-8001-c63ea7701731 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:08.309106 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:08.391898 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8a5e16-0442-44d5-889c-3d46d00c1e7b None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:08.400702 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cea7831-c658-4b66-8001-c63ea7701731 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:08.425718 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ee34de-cafb-4b8f-b19b-17214f475293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:08.426322 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ee34de-cafb-4b8f-b19b-17214f475293 None admin] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:08.426604 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ee34de-cafb-4b8f-b19b-17214f475293 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:08.426954 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ee34de-cafb-4b8f-b19b-17214f475293 None admin] PATH_INFO: `/v3/users` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:08.427520 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ee34de-cafb-4b8f-b19b-17214f475293 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:08.428930 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ee34de-cafb-4b8f-b19b-17214f475293 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:08.434154 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16abe170-ab12-451b-bc26-989f77c480ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:08.435368 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16abe170-ab12-451b-bc26-989f77c480ee None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:08.435645 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16abe170-ab12-451b-bc26-989f77c480ee None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:08.435914 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16abe170-ab12-451b-bc26-989f77c480ee None admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:08.436550 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16abe170-ab12-451b-bc26-989f77c480ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:08.436851 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16abe170-ab12-451b-bc26-989f77c480ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:08.438214 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16abe170-ab12-451b-bc26-989f77c480ee None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:08.449267 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 262/526] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 24 17:08:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:08.450237 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16abe170-ab12-451b-bc26-989f77c480ee None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:08.512919 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:08.537584 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 265/527] 10.222.0.18 () {60 vars in 1132 bytes} [Wed Aug 24 17:08:08 2022] POST /identity/v3/users => generated 247 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:08:08.538400 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ee34de-cafb-4b8f-b19b-17214f475293 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:08.549770 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16abe170-ab12-451b-bc26-989f77c480ee None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:08.580685 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:08.590622 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1703a165-e6dc-4ac8-98c3-c28bd7d94dbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:08.591458 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1703a165-e6dc-4ac8-98c3-c28bd7d94dbc None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:08.591781 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1703a165-e6dc-4ac8-98c3-c28bd7d94dbc None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:08.594136 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1703a165-e6dc-4ac8-98c3-c28bd7d94dbc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:08.604274 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1703a165-e6dc-4ac8-98c3-c28bd7d94dbc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:08.606206 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1703a165-e6dc-4ac8-98c3-c28bd7d94dbc None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:08.623810 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-1703a165-e6dc-4ac8-98c3-c28bd7d94dbc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:08:08.623810 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:08.623810 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:08.623810 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:08.623810 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:08.623810 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:08.623810 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:08.623810 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:08.623810 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:08.623810 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:08.623810 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:08.623810 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:08.623810 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:08:08.623810 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:08:08.623810 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:08:08.623810 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:08:08.625001 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:08.625001 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:08.625001 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:08.625001 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:08.625001 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:08.625001 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:08.625001 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:08.625001 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:08.625001 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:08.625001 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:08.625001 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:08.625001 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:08.625001 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:08.625001 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:08.625001 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:08.625001 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:08.625001 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:08:08.626312 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:08:08.626312 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:08:08.626312 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:08:08.627350 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 263/528] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 24 17:08:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:08.628339 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1703a165-e6dc-4ac8-98c3-c28bd7d94dbc None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:08.680832 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ee34de-cafb-4b8f-b19b-17214f475293 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:08.719072 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e2c1e3-17ad-4a65-a718-ade10689ed5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:08.719605 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e2c1e3-17ad-4a65-a718-ade10689ed5f None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:08.719954 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e2c1e3-17ad-4a65-a718-ade10689ed5f None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:08.720230 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e2c1e3-17ad-4a65-a718-ade10689ed5f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:08.721531 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e2c1e3-17ad-4a65-a718-ade10689ed5f None admin] RBAC: Adding query filter params (role.id=e7db2a847f56438bac13d0ff21c96b73, scope.project.id=46ab158bc3644c888557781b0630b0b8, user.id=557aca9255cb4bbb8af52e796c5c5177) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:08.721531 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e2c1e3-17ad-4a65-a718-ade10689ed5f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:08.724524 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e2c1e3-17ad-4a65-a718-ade10689ed5f None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:08.729397 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 266/529] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 24 17:08:08 2022] GET /identity/v3/role_assignments?user.id=557aca9255cb4bbb8af52e796c5c5177&scope.project.id=46ab158bc3644c888557781b0630b0b8&role.id=e7db2a847f56438bac13d0ff21c96b73 => generated 259 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:08.730290 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e2c1e3-17ad-4a65-a718-ade10689ed5f None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:08.746675 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1703a165-e6dc-4ac8-98c3-c28bd7d94dbc None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:08.776635 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:08.799006 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05355aca-2c28-4c19-a06f-c6a773ba31a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:08.799365 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05355aca-2c28-4c19-a06f-c6a773ba31a6 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:08.799610 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05355aca-2c28-4c19-a06f-c6a773ba31a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:08.799946 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05355aca-2c28-4c19-a06f-c6a773ba31a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:08.811891 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05355aca-2c28-4c19-a06f-c6a773ba31a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:08.814868 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05355aca-2c28-4c19-a06f-c6a773ba31a6 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:08.833483 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-05355aca-2c28-4c19-a06f-c6a773ba31a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:08:08.833483 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:08.833483 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:08.833483 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:08.833483 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:08.833483 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:08.833483 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:08.833483 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:08.833483 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:08.833483 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:08.833483 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:08.833483 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:08.833483 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:08:08.833483 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:08:08.833483 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:08:08.833483 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:08:08.834667 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:08.834667 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:08.834667 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:08.834667 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:08.834667 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:08.834667 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:08.834667 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:08.834667 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:08.834667 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:08.834667 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:08.834667 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:08.834667 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:08.834667 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:08.834667 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:08.834667 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:08.834667 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:08.834667 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:08:08.835603 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:08:08.835603 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:08:08.835603 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:08:08.837292 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 264/530] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 24 17:08:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:08.838354 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05355aca-2c28-4c19-a06f-c6a773ba31a6 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:08.866927 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:08.889507 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e2c1e3-17ad-4a65-a718-ade10689ed5f None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:08.936112 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc557e6-3bb3-4dc1-91ac-01a1cc1f9946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:08.937127 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc557e6-3bb3-4dc1-91ac-01a1cc1f9946 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:08.937413 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc557e6-3bb3-4dc1-91ac-01a1cc1f9946 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:08.937685 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc557e6-3bb3-4dc1-91ac-01a1cc1f9946 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:08.947408 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc557e6-3bb3-4dc1-91ac-01a1cc1f9946 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:08.948898 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc557e6-3bb3-4dc1-91ac-01a1cc1f9946 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:08.954820 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05355aca-2c28-4c19-a06f-c6a773ba31a6 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:08.965339 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-ccc557e6-3bb3-4dc1-91ac-01a1cc1f9946 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:08:08.965339 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:08.965339 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:08.965339 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:08.965339 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:08.965339 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:08.965339 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:08.965339 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:08.965339 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:08.965339 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:08.965339 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:08.965339 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:08.965339 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:08:08.965339 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:08:08.965339 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:08:08.966723 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:08:08.966723 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:08.966723 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:08.966723 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:08.966723 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:08.966723 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:08.966723 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:08.966723 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:08.966723 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:08.966723 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:08.966723 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:08.966723 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:08.966723 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:08.966723 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:08.966723 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:08.966723 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:08.966723 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:08.967591 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:08:08.967591 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:08:08.967591 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:08:08.967591 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:08:08.967591 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:08:08.967591 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:08:08.967591 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:08:08.967591 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:08:08.968007 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 267/531] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 24 17:08:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:08.968798 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc557e6-3bb3-4dc1-91ac-01a1cc1f9946 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:08.993912 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e1794e-928f-4609-9e42-2b784a92cefd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:08.994598 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e1794e-928f-4609-9e42-2b784a92cefd None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:08.995198 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e1794e-928f-4609-9e42-2b784a92cefd None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:08.995476 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e1794e-928f-4609-9e42-2b784a92cefd None admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:08.996128 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e1794e-928f-4609-9e42-2b784a92cefd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:08.996435 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e1794e-928f-4609-9e42-2b784a92cefd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:08.997880 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e1794e-928f-4609-9e42-2b784a92cefd None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:09.014998 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 265/532] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 24 17:08:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:09.015858 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e1794e-928f-4609-9e42-2b784a92cefd None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:09.055257 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:09.084692 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc557e6-3bb3-4dc1-91ac-01a1cc1f9946 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:09.121395 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763df958-9fd1-4984-b572-95d6f0c859c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:09.122651 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763df958-9fd1-4984-b572-95d6f0c859c9 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:09.123613 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763df958-9fd1-4984-b572-95d6f0c859c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:09.124067 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763df958-9fd1-4984-b572-95d6f0c859c9 None admin] PATH_INFO: `/v3/domains` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:09.125160 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763df958-9fd1-4984-b572-95d6f0c859c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:09.125903 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763df958-9fd1-4984-b572-95d6f0c859c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:09.128114 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763df958-9fd1-4984-b572-95d6f0c859c9 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:09.143598 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e1794e-928f-4609-9e42-2b784a92cefd None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:09.145477 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 268/533] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 24 17:08:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:09.147434 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763df958-9fd1-4984-b572-95d6f0c859c9 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:09.182576 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19679ee-5721-4fdb-bc6b-eb4cf8b6771c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:09.183426 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19679ee-5721-4fdb-bc6b-eb4cf8b6771c None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:09.183536 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19679ee-5721-4fdb-bc6b-eb4cf8b6771c None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:09.183785 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19679ee-5721-4fdb-bc6b-eb4cf8b6771c None admin] PATH_INFO: `/v3/projects` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:09.184628 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19679ee-5721-4fdb-bc6b-eb4cf8b6771c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:09.184914 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19679ee-5721-4fdb-bc6b-eb4cf8b6771c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:09.186344 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19679ee-5721-4fdb-bc6b-eb4cf8b6771c None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:09.191267 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:09.210264 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 266/534] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Aug 24 17:08:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:09.211208 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19679ee-5721-4fdb-bc6b-eb4cf8b6771c None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:09.283175 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763df958-9fd1-4984-b572-95d6f0c859c9 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:09.310536 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:09.322158 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19679ee-5721-4fdb-bc6b-eb4cf8b6771c None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:09.329403 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4baae5-2ba2-4d6e-890d-7202173d6363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:09.330215 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4baae5-2ba2-4d6e-890d-7202173d6363 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:09.331049 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4baae5-2ba2-4d6e-890d-7202173d6363 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:09.331468 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4baae5-2ba2-4d6e-890d-7202173d6363 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:09.341239 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4baae5-2ba2-4d6e-890d-7202173d6363 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:09.343180 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4baae5-2ba2-4d6e-890d-7202173d6363 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:09.359145 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-4f4baae5-2ba2-4d6e-890d-7202173d6363 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:08:09.359145 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:09.359145 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:09.359145 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:09.359145 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:09.359145 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:09.359145 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:09.359145 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:09.359145 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:09.359145 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:09.359145 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:09.359145 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:09.359145 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:08:09.359145 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:08:09.359145 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:08:09.360237 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:08:09.360237 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:09.360237 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:09.360237 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:09.360237 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:09.360237 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:09.360237 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:09.360237 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:09.360237 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:09.360237 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:09.360237 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:09.360237 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:09.360237 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:09.360237 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:09.360237 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:09.360237 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:09.360237 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:09.361278 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:08:09.361278 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:08:09.361278 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:08:09.361278 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:08:09.361278 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:08:09.361278 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:08:09.361278 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:08:09.361278 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:08:09.363314 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f041f3-1713-4be5-b441-d8bc3dcbdca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:09.363488 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 269/535] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 24 17:08:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:09.364421 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4baae5-2ba2-4d6e-890d-7202173d6363 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:09.368902 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f041f3-1713-4be5-b441-d8bc3dcbdca4 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:09.369135 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f041f3-1713-4be5-b441-d8bc3dcbdca4 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:09.369540 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f041f3-1713-4be5-b441-d8bc3dcbdca4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:09.378281 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f041f3-1713-4be5-b441-d8bc3dcbdca4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:09.380040 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f041f3-1713-4be5-b441-d8bc3dcbdca4 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:09.393392 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-89f041f3-1713-4be5-b441-d8bc3dcbdca4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:08:09.393392 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:09.393392 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:09.393392 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:09.393392 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:09.393392 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:09.393392 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:09.393392 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:09.393392 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:09.393392 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:09.393392 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:09.393392 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:09.393392 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:08:09.393392 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:08:09.393392 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:08:09.395932 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:08:09.395932 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:09.395932 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:09.395932 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:09.395932 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:09.395932 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:09.395932 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:09.395932 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:09.395932 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:09.395932 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:09.395932 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:09.395932 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:09.395932 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:09.395932 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:09.395932 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:09.395932 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:09.395932 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:09.396974 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:08:09.396974 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:08:09.396974 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:08:09.396974 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:08:09.396974 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:08:09.396974 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:08:09.396974 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:08:09.396974 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:08:09.396974 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 267/536] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 24 17:08:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:09.397717 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f041f3-1713-4be5-b441-d8bc3dcbdca4 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:09.455704 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:09.503298 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4baae5-2ba2-4d6e-890d-7202173d6363 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:09.523796 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f041f3-1713-4be5-b441-d8bc3dcbdca4 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:09.539350 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bf45ec-3e65-46df-a07d-55aa4ce19668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:09.540012 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bf45ec-3e65-46df-a07d-55aa4ce19668 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:09.540406 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bf45ec-3e65-46df-a07d-55aa4ce19668 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:09.540797 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bf45ec-3e65-46df-a07d-55aa4ce19668 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:09.541620 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bf45ec-3e65-46df-a07d-55aa4ce19668 None admin] RBAC: Adding query filter params (role.id=e7db2a847f56438bac13d0ff21c96b73, scope.project.id=46ab158bc3644c888557781b0630b0b8, user.id=3553e522363046f6828fd61b2a0642ed) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:09.542032 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bf45ec-3e65-46df-a07d-55aa4ce19668 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:09.544222 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bf45ec-3e65-46df-a07d-55aa4ce19668 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:09.552718 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 270/537] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 24 17:08:09 2022] GET /identity/v3/role_assignments?user.id=3553e522363046f6828fd61b2a0642ed&scope.project.id=46ab158bc3644c888557781b0630b0b8&role.id=e7db2a847f56438bac13d0ff21c96b73 => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:09.560385 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bf45ec-3e65-46df-a07d-55aa4ce19668 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:09.567897 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e5c297-7816-41fb-b5ff-effb479bef00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:09.568764 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e5c297-7816-41fb-b5ff-effb479bef00 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:09.569187 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e5c297-7816-41fb-b5ff-effb479bef00 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:09.569601 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e5c297-7816-41fb-b5ff-effb479bef00 None admin] PATH_INFO: `/v3/projects` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:09.570729 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e5c297-7816-41fb-b5ff-effb479bef00 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:09.571227 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e5c297-7816-41fb-b5ff-effb479bef00 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:09.573796 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e5c297-7816-41fb-b5ff-effb479bef00 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:09.589152 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 268/538] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Aug 24 17:08:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:09.595752 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e5c297-7816-41fb-b5ff-effb479bef00 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:09.611579 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:09.728113 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e5c297-7816-41fb-b5ff-effb479bef00 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:09.773089 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4992bf9-8a29-4356-907f-962e1249657b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:09.773695 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4992bf9-8a29-4356-907f-962e1249657b None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:09.773965 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4992bf9-8a29-4356-907f-962e1249657b None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:09.774233 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4992bf9-8a29-4356-907f-962e1249657b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:09.774983 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4992bf9-8a29-4356-907f-962e1249657b None admin] RBAC: Adding query filter params (role.id=e7db2a847f56438bac13d0ff21c96b73, scope.project.id=46ab158bc3644c888557781b0630b0b8, user.id=0e9ed4d47aef4140bfb2d3bb798cc559) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:09.775290 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4992bf9-8a29-4356-907f-962e1249657b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:09.776730 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4992bf9-8a29-4356-907f-962e1249657b None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:09.783268 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 269/539] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 24 17:08:09 2022] GET /identity/v3/role_assignments?user.id=0e9ed4d47aef4140bfb2d3bb798cc559&scope.project.id=46ab158bc3644c888557781b0630b0b8&role.id=e7db2a847f56438bac13d0ff21c96b73 => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:09.797309 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bf45ec-3e65-46df-a07d-55aa4ce19668 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:09.850337 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:09.859756 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a3e922-fac8-49e1-b957-4fea97336bbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:09.865608 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a3e922-fac8-49e1-b957-4fea97336bbf None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:09.865608 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a3e922-fac8-49e1-b957-4fea97336bbf None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:09.865608 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a3e922-fac8-49e1-b957-4fea97336bbf None admin] PATH_INFO: `/v3/projects` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:09.865608 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a3e922-fac8-49e1-b957-4fea97336bbf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:09.865608 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a3e922-fac8-49e1-b957-4fea97336bbf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:09.865608 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a3e922-fac8-49e1-b957-4fea97336bbf None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:09.879363 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 271/540] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Aug 24 17:08:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:09.885100 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4992bf9-8a29-4356-907f-962e1249657b None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:09.996245 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:10.025091 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4992bf9-8a29-4356-907f-962e1249657b None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:10.063833 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d581dbd7-7b4c-4ec6-aaa8-7357b118adab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:10.064349 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d581dbd7-7b4c-4ec6-aaa8-7357b118adab None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:10.064629 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d581dbd7-7b4c-4ec6-aaa8-7357b118adab None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:10.064902 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d581dbd7-7b4c-4ec6-aaa8-7357b118adab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:10.065636 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d581dbd7-7b4c-4ec6-aaa8-7357b118adab None admin] RBAC: Adding query filter params (role.id=e7db2a847f56438bac13d0ff21c96b73, scope.project.id=46ab158bc3644c888557781b0630b0b8, user.id=be06246d47584b529150ceb98a5cfd95) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:10.065938 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d581dbd7-7b4c-4ec6-aaa8-7357b118adab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:10.067426 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d581dbd7-7b4c-4ec6-aaa8-7357b118adab None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:10.073784 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 270/541] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 24 17:08:09 2022] GET /identity/v3/role_assignments?user.id=be06246d47584b529150ceb98a5cfd95&scope.project.id=46ab158bc3644c888557781b0630b0b8&role.id=e7db2a847f56438bac13d0ff21c96b73 => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:10.707171 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56b3017-866f-468f-b558-0289a625b606 None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:10.707838 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56b3017-866f-468f-b558-0289a625b606 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:10.707838 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56b3017-866f-468f-b558-0289a625b606 None None] PATH_INFO: `/` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:10.709328 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 272/542] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:08:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:08:10.714837 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3ae2c3-9a26-4552-92d0-d474afd189c2 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:10.715220 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3ae2c3-9a26-4552-92d0-d474afd189c2 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:10.715440 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3ae2c3-9a26-4552-92d0-d474afd189c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:10.750146 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-da3ae2c3-9a26-4552-92d0-d474afd189c2 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:08:10.751631 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-da3ae2c3-9a26-4552-92d0-d474afd189c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:08:10.844574 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 271/543] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:08:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 17:08:11.068100 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b33981b-6262-40ca-8157-a4792e6fc1f1 None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:11.068466 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b33981b-6262-40ca-8157-a4792e6fc1f1 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:11.068641 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b33981b-6262-40ca-8157-a4792e6fc1f1 None None] PATH_INFO: `/` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:11.070104 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 273/544] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:08:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:08:11.075959 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7020868a-051d-4d03-a841-60c558c44595 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:11.076399 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7020868a-051d-4d03-a841-60c558c44595 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:11.076499 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7020868a-051d-4d03-a841-60c558c44595 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:11.100321 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-7020868a-051d-4d03-a841-60c558c44595 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:08:11.101498 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-7020868a-051d-4d03-a841-60c558c44595 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:08:11.155886 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b33981b-6262-40ca-8157-a4792e6fc1f1 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:11.179081 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:11.193484 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 272/545] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:08:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:08:11.284363 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b33981b-6262-40ca-8157-a4792e6fc1f1 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:11.319130 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d95f05d-fdc4-4bf0-b3f9-63ee01d84aed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:11.319755 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d95f05d-fdc4-4bf0-b3f9-63ee01d84aed None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:11.320089 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d95f05d-fdc4-4bf0-b3f9-63ee01d84aed None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:11.320363 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d95f05d-fdc4-4bf0-b3f9-63ee01d84aed None admin] PATH_INFO: `/v3/roles/service` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:11.331518 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d95f05d-fdc4-4bf0-b3f9-63ee01d84aed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:08:11.336342 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-8d95f05d-fdc4-4bf0-b3f9-63ee01d84aed None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:08:11.336342 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:11.336342 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:11.336342 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:11.336342 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:11.336342 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:11.336342 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:11.336342 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:11.336342 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:11.336342 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:11.336342 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:11.336342 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:11.336342 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:08:11.336342 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:08:11.336342 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:08:11.336342 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:08:11.337238 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:08:11.337238 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:08:11.337238 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:08:11.337238 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:08:11.337238 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:11.337238 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:11.337238 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:11.337238 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:11.337238 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:11.337238 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:11.337238 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:11.337238 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:11.337238 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:11.337238 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:11.337238 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:11.337238 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:11.338035 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:11.338035 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:11.338035 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:11.338035 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:11.338035 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:08:11.338035 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:08:11.338035 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:08:11.338035 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:08:11.338035 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:08:11.338035 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:08:11.338035 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:08:11.338035 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:08:11.338035 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 274/546] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Aug 24 17:08:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:08:11.344896 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7020868a-051d-4d03-a841-60c558c44595 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:11.476817 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7020868a-051d-4d03-a841-60c558c44595 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:11.525882 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e03b32-4288-4618-a70f-a6edba46f41c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:11.526485 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e03b32-4288-4618-a70f-a6edba46f41c None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:11.526760 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e03b32-4288-4618-a70f-a6edba46f41c None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:11.527055 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e03b32-4288-4618-a70f-a6edba46f41c None admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:11.527715 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e03b32-4288-4618-a70f-a6edba46f41c None admin] RBAC: Adding query filter params (name=service) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:11.528015 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e03b32-4288-4618-a70f-a6edba46f41c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:11.529397 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e03b32-4288-4618-a70f-a6edba46f41c None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:11.541991 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 273/547] 10.222.0.18 () {56 vars in 1103 bytes} [Wed Aug 24 17:08:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:11.563320 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d95f05d-fdc4-4bf0-b3f9-63ee01d84aed None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:11.638458 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:11.676987 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e03b32-4288-4618-a70f-a6edba46f41c None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:11.700426 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d95f05d-fdc4-4bf0-b3f9-63ee01d84aed None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:11.736911 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4773a5-5e04-4d31-b127-5be03b92c61e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:11.738361 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4773a5-5e04-4d31-b127-5be03b92c61e None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:11.738844 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4773a5-5e04-4d31-b127-5be03b92c61e None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:11.739349 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4773a5-5e04-4d31-b127-5be03b92c61e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:11.751167 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4773a5-5e04-4d31-b127-5be03b92c61e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:11.752983 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4773a5-5e04-4d31-b127-5be03b92c61e None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:11.768630 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-cd4773a5-5e04-4d31-b127-5be03b92c61e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:08:11.768630 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:11.768630 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:11.768630 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:11.768630 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:11.768630 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:11.768630 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:11.768630 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:11.768630 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:11.768630 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:11.768630 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:11.768630 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:11.768630 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:08:11.768630 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:08:11.768630 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:08:11.768630 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:08:11.769560 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:11.769560 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:11.769560 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:11.769560 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:11.769560 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:11.769560 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:11.769560 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:11.769560 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:11.769560 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:11.769560 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:11.769560 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:11.769560 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:11.769560 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:11.769560 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:11.769560 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:11.769560 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:11.769560 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:08:11.770511 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:08:11.770511 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:08:11.770511 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:08:11.772461 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 275/548] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 24 17:08:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:11.778826 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4773a5-5e04-4d31-b127-5be03b92c61e None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:11.808876 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:11.884865 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4773a5-5e04-4d31-b127-5be03b92c61e None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:11.886671 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:11.901083 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e03b32-4288-4618-a70f-a6edba46f41c None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:11.922039 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ce63b6-ef0f-4b1d-8f48-a85ed5acd096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:11.922998 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ce63b6-ef0f-4b1d-8f48-a85ed5acd096 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:11.923474 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ce63b6-ef0f-4b1d-8f48-a85ed5acd096 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:11.924334 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ce63b6-ef0f-4b1d-8f48-a85ed5acd096 None admin] PATH_INFO: `/v3/domains` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:11.925240 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ce63b6-ef0f-4b1d-8f48-a85ed5acd096 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:11.925741 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ce63b6-ef0f-4b1d-8f48-a85ed5acd096 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:11.927320 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ce63b6-ef0f-4b1d-8f48-a85ed5acd096 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:11.940817 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 276/549] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 24 17:08:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:08:11.946428 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ce63b6-ef0f-4b1d-8f48-a85ed5acd096 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:11.978240 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fd6516-ae7c-4818-834f-856f382cd496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:11.978876 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fd6516-ae7c-4818-834f-856f382cd496 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:11.979205 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fd6516-ae7c-4818-834f-856f382cd496 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:11.979475 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fd6516-ae7c-4818-834f-856f382cd496 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:12.007979 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fd6516-ae7c-4818-834f-856f382cd496 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:08:12.013939 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-c7fd6516-ae7c-4818-834f-856f382cd496 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:08:12.013939 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:12.013939 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:12.013939 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:12.013939 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:12.013939 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:12.013939 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:12.013939 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:12.013939 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:12.013939 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:12.013939 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:12.013939 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:12.013939 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:08:12.013939 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:08:12.013939 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:08:12.013939 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:08:12.015062 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:08:12.015062 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:08:12.015062 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:08:12.015062 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:08:12.015062 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:12.015062 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:12.015062 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:12.015062 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:12.015062 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:12.015062 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:12.015062 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:12.015062 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:12.015062 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:12.015062 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:12.015062 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:12.015062 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:12.015898 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:12.015898 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:12.015898 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:12.015898 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:12.015898 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:08:12.015898 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:08:12.015898 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:08:12.015898 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:08:12.015898 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:08:12.015898 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:08:12.015898 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:08:12.015898 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:08:12.021628 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 274/550] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Aug 24 17:08:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 343 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:12.021628 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f815071e-a752-499b-9057-dc790db570ec None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:12.021829 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f815071e-a752-499b-9057-dc790db570ec None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:12.022012 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f815071e-a752-499b-9057-dc790db570ec None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:12.023518 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 275/551] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:08:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:08:12.024701 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f815071e-a752-499b-9057-dc790db570ec None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:12.059866 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ce63b6-ef0f-4b1d-8f48-a85ed5acd096 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:12.105166 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac1f6b9-19dd-4781-9a71-8fa694edfd50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:12.106489 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac1f6b9-19dd-4781-9a71-8fa694edfd50 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:12.107218 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac1f6b9-19dd-4781-9a71-8fa694edfd50 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:12.107874 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac1f6b9-19dd-4781-9a71-8fa694edfd50 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:12.119355 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac1f6b9-19dd-4781-9a71-8fa694edfd50 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:12.122489 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac1f6b9-19dd-4781-9a71-8fa694edfd50 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:12.135492 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f815071e-a752-499b-9057-dc790db570ec None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:12.139444 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-3ac1f6b9-19dd-4781-9a71-8fa694edfd50 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 17:08:12.139444 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:12.139444 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:12.139444 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:12.139444 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:12.139444 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:12.139444 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:12.139444 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:12.139444 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:12.139444 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:12.139444 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:12.139444 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:12.139444 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:08:12.139444 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:08:12.139444 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:08:12.139444 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:08:12.140478 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:12.140478 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:12.140478 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:08:12.140478 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:08:12.140478 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:08:12.140478 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:08:12.140478 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:12.140478 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:12.140478 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:12.140478 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:12.140478 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:12.140478 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:12.140478 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:12.140478 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:12.140478 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:12.140478 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:12.140478 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:12.140478 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:12.141416 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:12.141416 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:12.141416 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:08:12.141416 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:08:12.141416 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:08:12.141416 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:08:12.141416 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:08:12.141416 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:08:12.141416 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 17:08:12.141416 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:08:12.146311 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 277/552] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Aug 24 17:08:11 2022] GET /identity/v3/users/swift => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:08:12.148231 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545d504b-943c-4186-824a-1bfa30fbfbfa None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:12.148654 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545d504b-943c-4186-824a-1bfa30fbfbfa None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:12.149065 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545d504b-943c-4186-824a-1bfa30fbfbfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:12.157301 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:12.169598 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-545d504b-943c-4186-824a-1bfa30fbfbfa None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:08:12.170796 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-545d504b-943c-4186-824a-1bfa30fbfbfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:08:12.176692 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4af2bb-1b67-478e-a9ab-1bba8ffe5caf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:12.192384 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4af2bb-1b67-478e-a9ab-1bba8ffe5caf None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:12.192384 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4af2bb-1b67-478e-a9ab-1bba8ffe5caf None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:12.192384 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4af2bb-1b67-478e-a9ab-1bba8ffe5caf None admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:12.192384 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4af2bb-1b67-478e-a9ab-1bba8ffe5caf None admin] RBAC: Adding query filter params (name=service) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:12.192384 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4af2bb-1b67-478e-a9ab-1bba8ffe5caf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:12.192384 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4af2bb-1b67-478e-a9ab-1bba8ffe5caf None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:12.192797 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:12.192870 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 276/553] 10.222.0.18 () {56 vars in 1103 bytes} [Wed Aug 24 17:08:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:12.192870 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0a04ce-3f20-4e1c-9bd0-d2591745d81b None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:12.194095 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0a04ce-3f20-4e1c-9bd0-d2591745d81b None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:12.194380 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0a04ce-3f20-4e1c-9bd0-d2591745d81b None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:12.195838 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 277/554] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:08:12 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:08:12.196533 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0a04ce-3f20-4e1c-9bd0-d2591745d81b None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:12.245415 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 278/555] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:08:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:08:12.246535 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545d504b-943c-4186-824a-1bfa30fbfbfa None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:12.310499 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0a04ce-3f20-4e1c-9bd0-d2591745d81b None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:12.348209 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7b5917-1e10-42f4-8574-be6dae47004b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:12.348900 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7b5917-1e10-42f4-8574-be6dae47004b None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:12.349244 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7b5917-1e10-42f4-8574-be6dae47004b None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:12.349583 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7b5917-1e10-42f4-8574-be6dae47004b None admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:12.350386 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7b5917-1e10-42f4-8574-be6dae47004b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:12.350748 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7b5917-1e10-42f4-8574-be6dae47004b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:12.352261 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7b5917-1e10-42f4-8574-be6dae47004b None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:12.375027 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 278/556] 10.222.0.18 () {56 vars in 1135 bytes} [Wed Aug 24 17:08:12 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:12.375027 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab257b2-311c-42bc-8d7b-2061066bc831 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:12.375027 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab257b2-311c-42bc-8d7b-2061066bc831 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:12.375027 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab257b2-311c-42bc-8d7b-2061066bc831 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:12.391606 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-bab257b2-311c-42bc-8d7b-2061066bc831 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:08:12.392889 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-bab257b2-311c-42bc-8d7b-2061066bc831 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:08:12.393139 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545d504b-943c-4186-824a-1bfa30fbfbfa None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:12.431858 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:12.444182 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ff7135-dfc4-47e8-a021-017cae25a201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:12.446220 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ff7135-dfc4-47e8-a021-017cae25a201 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:12.447303 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ff7135-dfc4-47e8-a021-017cae25a201 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:12.447960 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ff7135-dfc4-47e8-a021-017cae25a201 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:12.467370 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 279/557] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:08:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:08:12.468391 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab257b2-311c-42bc-8d7b-2061066bc831 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:12.470983 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ff7135-dfc4-47e8-a021-017cae25a201 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:12.473786 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ff7135-dfc4-47e8-a021-017cae25a201 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:12.497755 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-a7ff7135-dfc4-47e8-a021-017cae25a201 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:08:12.497755 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:12.497755 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:12.497755 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:12.497755 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:12.497755 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:12.497755 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:12.497755 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:12.497755 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:12.497755 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:12.497755 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:12.497755 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:12.497755 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:08:12.497755 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:08:12.497755 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:08:12.497755 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:08:12.498758 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:12.498758 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:12.498758 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:12.498758 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:12.498758 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:12.498758 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:12.498758 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:12.498758 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:12.498758 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:12.498758 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:12.498758 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:12.498758 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:12.498758 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:12.498758 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:12.498758 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:12.498758 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:12.498758 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:08:12.499749 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:08:12.499749 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:08:12.499749 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:08:12.500523 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 279/558] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 24 17:08:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:12.508265 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ff7135-dfc4-47e8-a021-017cae25a201 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:12.512610 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:12.596198 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab257b2-311c-42bc-8d7b-2061066bc831 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:12.636054 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a12ef74-bf29-4975-8aa6-6e878a0be93c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:12.636667 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a12ef74-bf29-4975-8aa6-6e878a0be93c None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:12.636949 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a12ef74-bf29-4975-8aa6-6e878a0be93c None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:12.637455 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a12ef74-bf29-4975-8aa6-6e878a0be93c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:12.649507 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ff7135-dfc4-47e8-a021-017cae25a201 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:12.683799 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:12.689904 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8083fa-d64a-4518-a00f-691ad7ed3132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:12.690507 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8083fa-d64a-4518-a00f-691ad7ed3132 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:12.694191 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8083fa-d64a-4518-a00f-691ad7ed3132 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:12.694191 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8083fa-d64a-4518-a00f-691ad7ed3132 None admin] PATH_INFO: `/v3/domains` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:12.694191 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8083fa-d64a-4518-a00f-691ad7ed3132 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:12.694191 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8083fa-d64a-4518-a00f-691ad7ed3132 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:12.694191 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8083fa-d64a-4518-a00f-691ad7ed3132 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:12.710302 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 280/559] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 24 17:08:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:12.710302 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d656810-094f-4a81-a8b4-a418fb3016a6 None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:12.710302 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d656810-094f-4a81-a8b4-a418fb3016a6 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:12.710302 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d656810-094f-4a81-a8b4-a418fb3016a6 None None] PATH_INFO: `/` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:12.710302 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 281/560] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:08:12 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:08:12.710302 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d656810-094f-4a81-a8b4-a418fb3016a6 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:12.717282 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a12ef74-bf29-4975-8aa6-6e878a0be93c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:12.719305 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a12ef74-bf29-4975-8aa6-6e878a0be93c None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:12.775023 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:12.815060 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d656810-094f-4a81-a8b4-a418fb3016a6 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:12.819505 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 280/561] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 24 17:08:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 17:08:12.820334 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a12ef74-bf29-4975-8aa6-6e878a0be93c None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:12.850739 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4218f9f-f95e-428b-9447-8054287ce585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:12.851386 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4218f9f-f95e-428b-9447-8054287ce585 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:12.851664 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4218f9f-f95e-428b-9447-8054287ce585 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:12.851934 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4218f9f-f95e-428b-9447-8054287ce585 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:12.861732 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4218f9f-f95e-428b-9447-8054287ce585 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:08:12.866364 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-f4218f9f-f95e-428b-9447-8054287ce585 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:08:12.866364 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:12.866364 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:12.866364 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:12.866364 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:12.866364 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:12.866364 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:12.866364 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:12.866364 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:12.866364 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:12.866364 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:12.866364 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:12.866364 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:08:12.866364 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:08:12.866364 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:08:12.866364 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:08:12.867454 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:08:12.867454 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:08:12.867454 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:08:12.867454 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:08:12.867454 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:12.867454 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:12.867454 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:12.867454 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:12.867454 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:12.867454 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:12.867454 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:12.867454 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:12.867454 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:12.867454 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:12.867454 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:12.867454 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:12.868281 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:12.868281 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:12.868281 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:12.868281 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:12.868281 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:08:12.868281 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:08:12.868281 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:08:12.868281 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:08:12.868281 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:08:12.868281 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:08:12.868281 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:08:12.868281 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:08:12.871038 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 282/562] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Aug 24 17:08:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:12.872946 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e02531-aabf-4d8d-81a8-14028ab1f9a8 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:12.873228 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e02531-aabf-4d8d-81a8-14028ab1f9a8 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:12.873499 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e02531-aabf-4d8d-81a8-14028ab1f9a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:12.890539 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:12.891070 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-83e02531-aabf-4d8d-81a8-14028ab1f9a8 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:08:12.892252 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-83e02531-aabf-4d8d-81a8-14028ab1f9a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:08:12.920414 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a12ef74-bf29-4975-8aa6-6e878a0be93c None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:12.954073 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b425801d-adb9-4799-845b-78f6258ead93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:12.954776 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b425801d-adb9-4799-845b-78f6258ead93 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:12.955093 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b425801d-adb9-4799-845b-78f6258ead93 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:12.955362 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b425801d-adb9-4799-845b-78f6258ead93 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:12.956542 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 283/563] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:08:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:08:12.957282 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e02531-aabf-4d8d-81a8-14028ab1f9a8 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:12.962374 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b425801d-adb9-4799-845b-78f6258ead93 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:12.963853 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b425801d-adb9-4799-845b-78f6258ead93 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:12.975647 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-b425801d-adb9-4799-845b-78f6258ead93 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 17:08:12.975647 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:12.975647 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:12.975647 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:12.975647 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:12.975647 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:12.975647 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:12.975647 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:12.975647 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:12.975647 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:12.975647 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:12.975647 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:12.975647 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:08:12.975647 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:08:12.975647 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:08:12.975647 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:08:12.976667 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:12.976667 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:12.976667 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:08:12.976667 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:08:12.976667 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:08:12.976667 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:08:12.976667 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:12.976667 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:12.976667 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:12.976667 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:12.976667 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:12.976667 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:12.976667 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:12.976667 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:12.976667 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:12.976667 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:12.976667 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:12.976667 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:12.977613 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:12.977613 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:12.977613 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:08:12.977613 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:08:12.977613 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:08:12.977613 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:08:12.977613 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:08:12.977613 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:08:12.977613 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 17:08:12.977613 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:08:12.977613 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 281/564] 10.222.0.18 () {56 vars in 1093 bytes} [Wed Aug 24 17:08:12 2022] GET /identity/v3/users/nova => generated 82 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:12.977613 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b425801d-adb9-4799-845b-78f6258ead93 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:13.021495 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:13.067262 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e02531-aabf-4d8d-81a8-14028ab1f9a8 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:13.083869 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b425801d-adb9-4799-845b-78f6258ead93 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:13.100320 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cded6b-4960-4c58-a6ab-5cb3edfd76dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:13.101003 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cded6b-4960-4c58-a6ab-5cb3edfd76dd None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:13.101224 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cded6b-4960-4c58-a6ab-5cb3edfd76dd None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:13.101496 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cded6b-4960-4c58-a6ab-5cb3edfd76dd None admin] PATH_INFO: `/v3/roles/service` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:13.112121 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cded6b-4960-4c58-a6ab-5cb3edfd76dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:08:13.112944 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:13.118110 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-27cded6b-4960-4c58-a6ab-5cb3edfd76dd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:08:13.118110 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:13.118110 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:13.118110 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:13.118110 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:13.118110 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:13.118110 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:13.118110 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:13.118110 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:13.118110 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:13.118110 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:13.118110 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:13.118110 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:08:13.118110 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:08:13.118110 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:08:13.118110 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:08:13.119139 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:08:13.119139 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:08:13.119139 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:08:13.119139 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:08:13.119139 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:13.119139 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:13.119139 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:13.119139 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:13.119139 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:13.119139 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:13.119139 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:13.119139 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:13.119139 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:13.119139 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:13.119139 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:13.119139 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:13.120002 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:13.120002 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:13.120002 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:13.120002 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:13.120002 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:08:13.120002 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:08:13.120002 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:08:13.120002 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:08:13.120002 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:08:13.120002 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:08:13.120002 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:08:13.120002 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:08:13.120002 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 284/565] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Aug 24 17:08:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:13.120002 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cded6b-4960-4c58-a6ab-5cb3edfd76dd None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:13.125445 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af778935-013e-4daa-af0c-97003d1517e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:13.127270 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af778935-013e-4daa-af0c-97003d1517e5 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:13.127554 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af778935-013e-4daa-af0c-97003d1517e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:13.127852 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af778935-013e-4daa-af0c-97003d1517e5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:13.138857 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af778935-013e-4daa-af0c-97003d1517e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:13.140338 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af778935-013e-4daa-af0c-97003d1517e5 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:13.158786 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-af778935-013e-4daa-af0c-97003d1517e5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:08:13.158786 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:13.158786 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:13.158786 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:13.158786 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:13.158786 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:13.158786 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:13.158786 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:13.158786 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:13.158786 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:13.158786 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:13.158786 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:13.158786 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:08:13.158786 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:08:13.158786 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:08:13.158786 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:08:13.159857 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:13.159857 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:13.159857 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:13.159857 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:13.159857 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:13.159857 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:13.159857 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:13.159857 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:13.159857 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:13.159857 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:13.159857 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:13.159857 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:13.159857 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:13.159857 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:13.159857 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:13.159857 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:13.159857 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:08:13.165667 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:08:13.165667 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:08:13.165667 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:08:13.165667 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 282/566] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 24 17:08:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:13.165667 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af778935-013e-4daa-af0c-97003d1517e5 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:13.229908 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cded6b-4960-4c58-a6ab-5cb3edfd76dd None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:13.243504 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:13.263977 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3287c46a-0e82-49de-a212-a46bae80e261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:13.264580 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3287c46a-0e82-49de-a212-a46bae80e261 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:13.264861 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3287c46a-0e82-49de-a212-a46bae80e261 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:13.265135 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3287c46a-0e82-49de-a212-a46bae80e261 None admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:13.265794 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3287c46a-0e82-49de-a212-a46bae80e261 None admin] RBAC: Adding query filter params (name=service) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:13.266100 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3287c46a-0e82-49de-a212-a46bae80e261 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:13.267496 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3287c46a-0e82-49de-a212-a46bae80e261 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:13.268968 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af778935-013e-4daa-af0c-97003d1517e5 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:13.276246 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 285/567] 10.222.0.18 () {56 vars in 1103 bytes} [Wed Aug 24 17:08:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:13.280040 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3287c46a-0e82-49de-a212-a46bae80e261 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:13.303187 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612cc6d0-6a11-47c7-83c0-00a56d693101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:13.303784 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612cc6d0-6a11-47c7-83c0-00a56d693101 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:13.304060 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612cc6d0-6a11-47c7-83c0-00a56d693101 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:13.304331 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612cc6d0-6a11-47c7-83c0-00a56d693101 None admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:13.305049 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612cc6d0-6a11-47c7-83c0-00a56d693101 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:13.305349 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612cc6d0-6a11-47c7-83c0-00a56d693101 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:13.306760 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612cc6d0-6a11-47c7-83c0-00a56d693101 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:13.323509 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 283/568] 10.222.0.18 () {56 vars in 1133 bytes} [Wed Aug 24 17:08:13 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:13.324289 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612cc6d0-6a11-47c7-83c0-00a56d693101 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:13.365008 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:13.383256 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3287c46a-0e82-49de-a212-a46bae80e261 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:13.422233 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612cc6d0-6a11-47c7-83c0-00a56d693101 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:13.445635 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3914be-5e40-4ca4-9905-6a155f7dedce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:13.446242 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3914be-5e40-4ca4-9905-6a155f7dedce None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:13.446521 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3914be-5e40-4ca4-9905-6a155f7dedce None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:13.446790 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3914be-5e40-4ca4-9905-6a155f7dedce None admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:13.447482 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3914be-5e40-4ca4-9905-6a155f7dedce None admin] RBAC: Adding query filter params (name=service) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:13.447780 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3914be-5e40-4ca4-9905-6a155f7dedce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:13.449135 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3914be-5e40-4ca4-9905-6a155f7dedce None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:13.457678 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 286/569] 10.222.0.18 () {56 vars in 1103 bytes} [Wed Aug 24 17:08:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:13.458485 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3914be-5e40-4ca4-9905-6a155f7dedce None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:13.463959 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7a52ab-cf3a-41d8-9222-2f7ccfe9fb92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:13.464568 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7a52ab-cf3a-41d8-9222-2f7ccfe9fb92 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:13.464847 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7a52ab-cf3a-41d8-9222-2f7ccfe9fb92 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:13.465120 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7a52ab-cf3a-41d8-9222-2f7ccfe9fb92 None admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:13.466408 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7a52ab-cf3a-41d8-9222-2f7ccfe9fb92 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:13.466708 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7a52ab-cf3a-41d8-9222-2f7ccfe9fb92 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:13.468518 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7a52ab-cf3a-41d8-9222-2f7ccfe9fb92 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:13.481251 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 284/570] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 24 17:08:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:13.482714 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7a52ab-cf3a-41d8-9222-2f7ccfe9fb92 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:13.501772 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:13.559951 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3914be-5e40-4ca4-9905-6a155f7dedce None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:13.585759 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:13.588645 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7a52ab-cf3a-41d8-9222-2f7ccfe9fb92 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:13.592955 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b96421-edc7-4e0d-bfef-10c568bee722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:13.593580 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b96421-edc7-4e0d-bfef-10c568bee722 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:13.593856 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b96421-edc7-4e0d-bfef-10c568bee722 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:13.594125 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b96421-edc7-4e0d-bfef-10c568bee722 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:13.604083 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b96421-edc7-4e0d-bfef-10c568bee722 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:08:13.609925 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-60b96421-edc7-4e0d-bfef-10c568bee722 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:08:13.609925 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:13.609925 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:13.609925 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:13.609925 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:13.609925 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:13.609925 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:13.609925 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:13.609925 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:13.609925 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:13.609925 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:13.609925 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:13.609925 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:08:13.609925 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:08:13.609925 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:08:13.609925 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:08:13.611033 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:08:13.611033 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:08:13.611033 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:08:13.611033 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:08:13.611033 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:13.611033 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:13.611033 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:13.611033 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:13.611033 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:13.611033 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:13.611033 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:13.611033 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:13.611033 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:13.611033 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:13.611033 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:13.611033 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:13.611879 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:13.611879 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:13.611879 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:13.611879 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:13.611879 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:08:13.611879 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:08:13.611879 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:08:13.611879 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:08:13.611879 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:08:13.611879 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:08:13.611879 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:08:13.611879 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:08:13.611879 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 287/571] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Aug 24 17:08:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:13.611879 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b96421-edc7-4e0d-bfef-10c568bee722 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:13.625066 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded5bb41-197b-4db4-9eb2-9aec11b6abe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:13.625887 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded5bb41-197b-4db4-9eb2-9aec11b6abe7 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:13.626172 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded5bb41-197b-4db4-9eb2-9aec11b6abe7 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:13.626444 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded5bb41-197b-4db4-9eb2-9aec11b6abe7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:13.636819 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded5bb41-197b-4db4-9eb2-9aec11b6abe7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:13.638760 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded5bb41-197b-4db4-9eb2-9aec11b6abe7 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:13.652814 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-ded5bb41-197b-4db4-9eb2-9aec11b6abe7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:08:13.652814 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:13.652814 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:13.652814 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:13.652814 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:13.652814 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:13.652814 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:13.652814 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:13.652814 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:13.652814 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:13.652814 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:13.652814 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:13.652814 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:08:13.652814 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:08:13.652814 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:08:13.652814 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:08:13.654332 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:13.654332 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:13.654332 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:13.654332 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:13.654332 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:13.654332 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:13.654332 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:13.654332 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:13.654332 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:13.654332 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:13.654332 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:13.654332 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:13.654332 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:13.654332 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:13.654332 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:13.654332 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:13.654332 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:08:13.655242 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:08:13.655242 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:08:13.655242 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:08:13.655242 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 285/572] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 24 17:08:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:13.655464 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded5bb41-197b-4db4-9eb2-9aec11b6abe7 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:13.710045 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:13.713440 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b96421-edc7-4e0d-bfef-10c568bee722 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:13.745886 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770475f4-e5f9-45ff-9a04-e93c8665bb4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:13.746503 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770475f4-e5f9-45ff-9a04-e93c8665bb4d None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:13.746816 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770475f4-e5f9-45ff-9a04-e93c8665bb4d None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:13.747632 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770475f4-e5f9-45ff-9a04-e93c8665bb4d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:13.753712 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded5bb41-197b-4db4-9eb2-9aec11b6abe7 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:13.758570 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770475f4-e5f9-45ff-9a04-e93c8665bb4d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:13.760042 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770475f4-e5f9-45ff-9a04-e93c8665bb4d None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:13.775830 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-770475f4-e5f9-45ff-9a04-e93c8665bb4d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:08:13.775830 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:13.775830 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:13.775830 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:13.775830 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:13.775830 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:13.775830 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:13.775830 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:13.775830 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:13.775830 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:13.775830 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:13.775830 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:13.775830 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:08:13.775830 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:08:13.775830 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:08:13.775830 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:08:13.777360 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:13.777360 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:13.777360 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:13.777360 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:13.777360 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:13.777360 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:13.777360 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:13.777360 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:13.777360 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:13.777360 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:13.777360 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:13.777360 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:13.777360 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:13.777360 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:13.777360 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:13.777360 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:13.777360 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:08:13.778882 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:08:13.778882 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:08:13.778882 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:08:13.778882 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 288/573] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 24 17:08:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:13.778882 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770475f4-e5f9-45ff-9a04-e93c8665bb4d None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:13.793251 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8462bb-c5ed-4765-b043-2b0fe71af998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:13.795472 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8462bb-c5ed-4765-b043-2b0fe71af998 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:13.795472 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8462bb-c5ed-4765-b043-2b0fe71af998 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:13.795472 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8462bb-c5ed-4765-b043-2b0fe71af998 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:13.804979 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8462bb-c5ed-4765-b043-2b0fe71af998 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:13.806431 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8462bb-c5ed-4765-b043-2b0fe71af998 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:13.820731 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-3f8462bb-c5ed-4765-b043-2b0fe71af998 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:08:13.820731 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:13.820731 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:13.820731 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:13.820731 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:13.820731 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:13.820731 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:13.820731 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:13.820731 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:13.820731 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:13.820731 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:13.820731 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:13.820731 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:08:13.820731 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:08:13.820731 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:08:13.820731 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:08:13.821667 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:13.821667 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:13.821667 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:13.821667 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:13.821667 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:13.821667 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:13.821667 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:13.821667 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:13.821667 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:13.821667 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:13.821667 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:13.821667 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:13.821667 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:13.821667 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:13.821667 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:13.821667 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:13.821667 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:08:13.822592 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:08:13.822592 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:08:13.822592 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:08:13.822592 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 286/574] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 24 17:08:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:13.822592 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8462bb-c5ed-4765-b043-2b0fe71af998 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:13.826433 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:13.883563 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770475f4-e5f9-45ff-9a04-e93c8665bb4d None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:13.916284 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a242dc58-44b2-4bea-9de4-72bb90b6c20e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:13.916908 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a242dc58-44b2-4bea-9de4-72bb90b6c20e None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:13.917188 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a242dc58-44b2-4bea-9de4-72bb90b6c20e None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:13.917466 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a242dc58-44b2-4bea-9de4-72bb90b6c20e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:13.920097 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8462bb-c5ed-4765-b043-2b0fe71af998 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:13.924066 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a242dc58-44b2-4bea-9de4-72bb90b6c20e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:13.925511 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a242dc58-44b2-4bea-9de4-72bb90b6c20e None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:13.936427 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-a242dc58-44b2-4bea-9de4-72bb90b6c20e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:08:13.936427 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:13.936427 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:13.936427 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:13.936427 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:13.936427 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:13.936427 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:13.936427 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:13.936427 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:13.936427 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:13.936427 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:13.936427 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:13.936427 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:08:13.936427 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:08:13.936427 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:08:13.937403 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:08:13.937403 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:13.937403 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:13.937403 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:13.937403 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:13.937403 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:13.937403 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:13.937403 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:13.937403 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:13.937403 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:13.937403 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:13.937403 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:13.937403 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:13.937403 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:13.937403 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:13.937403 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:13.937403 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:13.938309 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:08:13.938309 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:08:13.938309 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:08:13.938309 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:08:13.938309 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:08:13.938309 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:08:13.938309 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:08:13.938309 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:08:13.938309 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 289/575] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 24 17:08:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:13.938309 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a242dc58-44b2-4bea-9de4-72bb90b6c20e None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:13.951167 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:13.953520 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f07a680-f47a-42bf-922a-5c5922570ea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:13.954105 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f07a680-f47a-42bf-922a-5c5922570ea8 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:13.954382 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f07a680-f47a-42bf-922a-5c5922570ea8 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:13.954649 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f07a680-f47a-42bf-922a-5c5922570ea8 None admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:13.955340 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f07a680-f47a-42bf-922a-5c5922570ea8 None admin] RBAC: Adding query filter params (name=service) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:13.955639 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f07a680-f47a-42bf-922a-5c5922570ea8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:13.956981 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f07a680-f47a-42bf-922a-5c5922570ea8 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:13.964680 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 287/576] 10.222.0.18 () {56 vars in 1103 bytes} [Wed Aug 24 17:08:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:13.965537 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f07a680-f47a-42bf-922a-5c5922570ea8 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:14.032337 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:14.036651 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a242dc58-44b2-4bea-9de4-72bb90b6c20e None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:14.066747 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f07a680-f47a-42bf-922a-5c5922570ea8 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:14.069297 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a8f222-454f-44f3-bfd1-23df4e3b6e96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:14.069895 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a8f222-454f-44f3-bfd1-23df4e3b6e96 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:14.070171 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a8f222-454f-44f3-bfd1-23df4e3b6e96 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:14.070441 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a8f222-454f-44f3-bfd1-23df4e3b6e96 None admin] PATH_INFO: `/v3/domains` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:14.071103 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a8f222-454f-44f3-bfd1-23df4e3b6e96 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:14.071401 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a8f222-454f-44f3-bfd1-23df4e3b6e96 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:14.072757 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a8f222-454f-44f3-bfd1-23df4e3b6e96 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:14.085573 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 290/577] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 24 17:08:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:14.086804 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a8f222-454f-44f3-bfd1-23df4e3b6e96 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:14.103227 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f6187c-1b64-4ec1-a1cd-47b51693a507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:14.103834 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f6187c-1b64-4ec1-a1cd-47b51693a507 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:14.104112 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f6187c-1b64-4ec1-a1cd-47b51693a507 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:14.104389 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f6187c-1b64-4ec1-a1cd-47b51693a507 None admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:14.105032 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f6187c-1b64-4ec1-a1cd-47b51693a507 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:14.105330 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f6187c-1b64-4ec1-a1cd-47b51693a507 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:14.106840 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f6187c-1b64-4ec1-a1cd-47b51693a507 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:14.118182 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 288/578] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 24 17:08:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:14.118998 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f6187c-1b64-4ec1-a1cd-47b51693a507 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:14.157565 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:14.186303 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a8f222-454f-44f3-bfd1-23df4e3b6e96 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:14.217365 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f6187c-1b64-4ec1-a1cd-47b51693a507 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:14.219610 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d609da-f3a7-4d14-a67e-87288bf2d39e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:14.220208 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d609da-f3a7-4d14-a67e-87288bf2d39e None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:14.220485 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d609da-f3a7-4d14-a67e-87288bf2d39e None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:14.220753 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d609da-f3a7-4d14-a67e-87288bf2d39e None admin] PATH_INFO: `/v3/domains` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:14.221381 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d609da-f3a7-4d14-a67e-87288bf2d39e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:14.221676 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d609da-f3a7-4d14-a67e-87288bf2d39e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:14.223061 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d609da-f3a7-4d14-a67e-87288bf2d39e None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:14.234652 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 291/579] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 24 17:08:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:14.235462 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d609da-f3a7-4d14-a67e-87288bf2d39e None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:14.243252 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:14.252134 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de70ddb8-cb6b-4eb4-9b6b-913e1db38d48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:14.252737 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de70ddb8-cb6b-4eb4-9b6b-913e1db38d48 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:14.253803 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de70ddb8-cb6b-4eb4-9b6b-913e1db38d48 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:14.253803 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de70ddb8-cb6b-4eb4-9b6b-913e1db38d48 None admin] PATH_INFO: `/v3/projects` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:14.254047 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de70ddb8-cb6b-4eb4-9b6b-913e1db38d48 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:14.254311 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de70ddb8-cb6b-4eb4-9b6b-913e1db38d48 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:14.255751 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de70ddb8-cb6b-4eb4-9b6b-913e1db38d48 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:14.267095 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 289/580] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Aug 24 17:08:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:14.267881 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de70ddb8-cb6b-4eb4-9b6b-913e1db38d48 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:14.335354 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d609da-f3a7-4d14-a67e-87288bf2d39e None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:14.364263 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:14.369954 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460679ce-28af-48e0-a999-8a8b7fa491ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:14.370581 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460679ce-28af-48e0-a999-8a8b7fa491ef None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:14.370858 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460679ce-28af-48e0-a999-8a8b7fa491ef None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:14.371160 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460679ce-28af-48e0-a999-8a8b7fa491ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:14.371258 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de70ddb8-cb6b-4eb4-9b6b-913e1db38d48 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:14.381646 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460679ce-28af-48e0-a999-8a8b7fa491ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:14.383227 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460679ce-28af-48e0-a999-8a8b7fa491ef None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:14.397469 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-460679ce-28af-48e0-a999-8a8b7fa491ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:08:14.397469 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:14.397469 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:14.397469 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:14.397469 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:14.397469 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:14.397469 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:14.397469 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:14.397469 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:14.397469 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:14.397469 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:14.397469 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:14.397469 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:08:14.397469 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:08:14.397469 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:08:14.397469 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:08:14.398406 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:14.398406 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:14.398406 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:14.398406 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:14.398406 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:14.398406 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:14.398406 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:14.398406 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:14.398406 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:14.398406 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:14.398406 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:14.398406 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:14.398406 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:14.398406 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:14.398406 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:14.398406 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:14.398406 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:08:14.399334 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:08:14.399334 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:08:14.399334 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:08:14.399334 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 292/581] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 24 17:08:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:14.399640 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460679ce-28af-48e0-a999-8a8b7fa491ef None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:14.408519 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa37680-8021-4822-a488-fee4f27b873a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:14.409091 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa37680-8021-4822-a488-fee4f27b873a None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:14.409367 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa37680-8021-4822-a488-fee4f27b873a None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:14.409637 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa37680-8021-4822-a488-fee4f27b873a None admin] PATH_INFO: `/v3/users/glance` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:14.417131 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa37680-8021-4822-a488-fee4f27b873a None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:14.418581 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa37680-8021-4822-a488-fee4f27b873a None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:14.429710 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-7aa37680-8021-4822-a488-fee4f27b873a None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 24 17:08:14.429710 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:14.429710 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:14.429710 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:14.429710 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:14.429710 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:14.429710 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:14.429710 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:14.429710 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:14.429710 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:14.429710 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:14.429710 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:14.429710 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:08:14.429710 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:08:14.429710 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:08:14.429710 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:08:14.430622 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:14.430622 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:14.430622 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:08:14.430622 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:08:14.430622 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:08:14.430622 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:08:14.430622 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:14.430622 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:14.430622 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:14.430622 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:14.430622 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:14.430622 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:14.430622 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:14.430622 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:14.430622 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:14.430622 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:14.430622 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:14.430622 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:14.432013 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:14.432013 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:14.432013 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:08:14.432013 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:08:14.432013 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:08:14.432013 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:08:14.432013 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:08:14.432013 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:08:14.432013 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 24 17:08:14.432013 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:08:14.432013 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 290/582] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 24 17:08:14 2022] GET /identity/v3/users/glance => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:14.432013 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa37680-8021-4822-a488-fee4f27b873a None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:14.479403 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:14.502408 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460679ce-28af-48e0-a999-8a8b7fa491ef None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:14.532331 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa37680-8021-4822-a488-fee4f27b873a None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:14.535484 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0881384c-f027-43c7-8441-41f9a868ede7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:14.536106 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0881384c-f027-43c7-8441-41f9a868ede7 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:14.536385 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0881384c-f027-43c7-8441-41f9a868ede7 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:14.536656 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0881384c-f027-43c7-8441-41f9a868ede7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:14.543072 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0881384c-f027-43c7-8441-41f9a868ede7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:14.546246 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0881384c-f027-43c7-8441-41f9a868ede7 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:14.557206 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-0881384c-f027-43c7-8441-41f9a868ede7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:08:14.557206 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:14.557206 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:14.557206 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:14.557206 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:14.557206 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:14.557206 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:14.557206 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:14.557206 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:14.557206 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:14.557206 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:14.557206 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:14.557206 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:08:14.557206 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:08:14.557206 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:08:14.558134 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:08:14.558134 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:14.558134 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:14.558134 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:14.558134 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:14.558134 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:14.558134 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:14.558134 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:14.558134 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:14.558134 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:14.558134 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:14.558134 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:14.558134 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:14.558134 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:14.558134 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:14.558134 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:14.558134 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:14.559068 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:08:14.559068 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:08:14.559068 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:08:14.559068 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:08:14.559068 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:08:14.559068 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:08:14.559068 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:08:14.559068 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:08:14.559068 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 293/583] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 24 17:08:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:14.559068 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0881384c-f027-43c7-8441-41f9a868ede7 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:14.571490 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4fd1fc-eb2e-4a9b-ad1f-5c2f9e24e7aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:14.572119 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4fd1fc-eb2e-4a9b-ad1f-5c2f9e24e7aa None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:14.572395 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4fd1fc-eb2e-4a9b-ad1f-5c2f9e24e7aa None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:14.572665 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4fd1fc-eb2e-4a9b-ad1f-5c2f9e24e7aa None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:14.580539 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4fd1fc-eb2e-4a9b-ad1f-5c2f9e24e7aa None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:14.581963 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4fd1fc-eb2e-4a9b-ad1f-5c2f9e24e7aa None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:14.593352 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-2a4fd1fc-eb2e-4a9b-ad1f-5c2f9e24e7aa None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 17:08:14.593352 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:14.593352 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:14.593352 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:14.593352 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:14.593352 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:14.593352 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:14.593352 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:14.593352 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:14.593352 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:14.593352 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:14.593352 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:14.593352 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:08:14.593352 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:08:14.593352 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:08:14.593352 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:08:14.594422 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:14.594422 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:14.594422 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:08:14.594422 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:08:14.594422 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:08:14.594422 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:08:14.594422 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:14.594422 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:14.594422 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:14.594422 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:14.594422 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:14.594422 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:14.594422 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:14.594422 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:14.594422 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:14.594422 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:14.594422 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:14.594422 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:14.595642 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:14.595642 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:14.595642 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:08:14.595642 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:08:14.595642 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:08:14.595642 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:08:14.595642 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:08:14.595642 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:08:14.595642 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 17:08:14.595642 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:08:14.595642 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 291/584] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Aug 24 17:08:14 2022] GET /identity/v3/users/neutron => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:14.595642 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4fd1fc-eb2e-4a9b-ad1f-5c2f9e24e7aa None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:14.603973 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:14.659617 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0881384c-f027-43c7-8441-41f9a868ede7 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:14.691648 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee11e11-a85e-4ab2-8be7-df4d4a7ed24a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:14.692240 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee11e11-a85e-4ab2-8be7-df4d4a7ed24a None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:14.692515 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee11e11-a85e-4ab2-8be7-df4d4a7ed24a None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:14.692781 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee11e11-a85e-4ab2-8be7-df4d4a7ed24a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:14.693488 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee11e11-a85e-4ab2-8be7-df4d4a7ed24a None admin] RBAC: Adding query filter params (role.id=e7db2a847f56438bac13d0ff21c96b73, scope.project.id=46ab158bc3644c888557781b0630b0b8, user.id=ef8caa353be34499a58cd25b72ee4032) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:14.693788 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee11e11-a85e-4ab2-8be7-df4d4a7ed24a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:14.695218 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee11e11-a85e-4ab2-8be7-df4d4a7ed24a None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:14.697875 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4fd1fc-eb2e-4a9b-ad1f-5c2f9e24e7aa None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:14.700846 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 294/585] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 24 17:08:14 2022] GET /identity/v3/role_assignments?user.id=ef8caa353be34499a58cd25b72ee4032&scope.project.id=46ab158bc3644c888557781b0630b0b8&role.id=e7db2a847f56438bac13d0ff21c96b73 => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:14.701614 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee11e11-a85e-4ab2-8be7-df4d4a7ed24a None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:14.705598 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:14.736203 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9229e72d-67cf-421d-bb79-67e2c15b252f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:14.737096 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9229e72d-67cf-421d-bb79-67e2c15b252f None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:14.737685 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9229e72d-67cf-421d-bb79-67e2c15b252f None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:14.738206 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9229e72d-67cf-421d-bb79-67e2c15b252f None admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:14.739347 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9229e72d-67cf-421d-bb79-67e2c15b252f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:14.739599 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9229e72d-67cf-421d-bb79-67e2c15b252f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:14.741000 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9229e72d-67cf-421d-bb79-67e2c15b252f None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:14.755729 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 292/586] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 24 17:08:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:14.756457 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9229e72d-67cf-421d-bb79-67e2c15b252f None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:14.812640 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee11e11-a85e-4ab2-8be7-df4d4a7ed24a None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:14.820271 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:14.851526 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bdd7c6-2c42-4433-8ff8-47cd09e4fe90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:14.852141 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bdd7c6-2c42-4433-8ff8-47cd09e4fe90 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:14.853454 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bdd7c6-2c42-4433-8ff8-47cd09e4fe90 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:14.853454 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bdd7c6-2c42-4433-8ff8-47cd09e4fe90 None admin] PATH_INFO: `/v3/users` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:14.853655 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bdd7c6-2c42-4433-8ff8-47cd09e4fe90 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:14.853841 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bdd7c6-2c42-4433-8ff8-47cd09e4fe90 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:14.855655 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bdd7c6-2c42-4433-8ff8-47cd09e4fe90 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:14.871335 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9229e72d-67cf-421d-bb79-67e2c15b252f None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:14.878134 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 295/587] 10.222.0.18 () {56 vars in 1137 bytes} [Wed Aug 24 17:08:14 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:14.878134 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bdd7c6-2c42-4433-8ff8-47cd09e4fe90 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:14.914132 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0431bc-a08d-4a71-91f4-a478efcd700f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:14.914660 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0431bc-a08d-4a71-91f4-a478efcd700f None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:14.914972 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0431bc-a08d-4a71-91f4-a478efcd700f None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:14.915246 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0431bc-a08d-4a71-91f4-a478efcd700f None admin] PATH_INFO: `/v3/projects` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:14.915941 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0431bc-a08d-4a71-91f4-a478efcd700f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:14.916359 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0431bc-a08d-4a71-91f4-a478efcd700f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:14.917790 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0431bc-a08d-4a71-91f4-a478efcd700f None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:14.935579 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 293/588] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Aug 24 17:08:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:14.936420 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0431bc-a08d-4a71-91f4-a478efcd700f None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:14.965251 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:14.978254 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bdd7c6-2c42-4433-8ff8-47cd09e4fe90 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:15.014098 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ff1fd0-daf8-4778-984a-701cad042fef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:15.014697 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ff1fd0-daf8-4778-984a-701cad042fef None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:15.015011 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ff1fd0-daf8-4778-984a-701cad042fef None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:15.015284 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ff1fd0-daf8-4778-984a-701cad042fef None admin] PATH_INFO: `/v3/users` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:15.016001 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ff1fd0-daf8-4778-984a-701cad042fef None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:15.016299 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ff1fd0-daf8-4778-984a-701cad042fef None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:15.017691 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ff1fd0-daf8-4778-984a-701cad042fef None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:15.036806 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 296/589] 10.222.0.18 () {56 vars in 1139 bytes} [Wed Aug 24 17:08:14 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:15.037629 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ff1fd0-daf8-4778-984a-701cad042fef None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:15.054836 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0431bc-a08d-4a71-91f4-a478efcd700f None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:15.067841 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:15.091407 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dced598-2380-460a-be33-beb8f649edc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:15.092039 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dced598-2380-460a-be33-beb8f649edc3 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:15.092321 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dced598-2380-460a-be33-beb8f649edc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:15.092597 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dced598-2380-460a-be33-beb8f649edc3 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:15.100219 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dced598-2380-460a-be33-beb8f649edc3 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:15.101663 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dced598-2380-460a-be33-beb8f649edc3 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:15.112710 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-0dced598-2380-460a-be33-beb8f649edc3 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 17:08:15.112710 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:15.112710 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:15.112710 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:15.112710 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:15.112710 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:15.112710 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:15.112710 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:15.112710 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:15.112710 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:15.112710 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:15.112710 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:15.112710 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:08:15.112710 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:08:15.112710 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:08:15.112710 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:08:15.113766 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:15.113766 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:15.113766 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:08:15.113766 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:08:15.113766 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:08:15.113766 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:08:15.113766 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:15.113766 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:15.113766 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:15.113766 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:15.113766 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:15.113766 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:15.113766 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:15.113766 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:15.113766 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:15.113766 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:15.113766 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:15.113766 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:15.114704 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:15.114704 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:15.114704 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:08:15.114704 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:08:15.114704 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:08:15.114704 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:08:15.114704 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:08:15.114704 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:08:15.114704 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 17:08:15.114704 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:08:15.114704 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 294/590] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 24 17:08:14 2022] GET /identity/v3/users/cinder => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:15.114704 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dced598-2380-460a-be33-beb8f649edc3 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:15.148256 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ff1fd0-daf8-4778-984a-701cad042fef None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:15.186841 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a314be1-1feb-4fb3-8509-c52d4ba476df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:15.187749 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a314be1-1feb-4fb3-8509-c52d4ba476df None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:15.188123 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a314be1-1feb-4fb3-8509-c52d4ba476df None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:15.188303 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a314be1-1feb-4fb3-8509-c52d4ba476df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:15.198550 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a314be1-1feb-4fb3-8509-c52d4ba476df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:15.200056 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a314be1-1feb-4fb3-8509-c52d4ba476df None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:15.211644 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:15.215358 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-1a314be1-1feb-4fb3-8509-c52d4ba476df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:08:15.215358 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:15.215358 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:15.215358 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:15.215358 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:15.215358 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:15.215358 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:15.215358 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:15.215358 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:15.215358 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:15.215358 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:15.215358 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:15.215358 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:08:15.215358 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:08:15.215358 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:08:15.215358 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:08:15.216300 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:15.216300 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:15.216300 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:15.216300 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:15.216300 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:15.216300 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:15.216300 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:15.216300 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:15.216300 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:15.216300 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:15.216300 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:15.216300 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:15.216300 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:15.216300 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:15.216300 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:15.216300 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:15.216300 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:08:15.217239 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:08:15.217239 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:08:15.217239 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:08:15.217239 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 297/591] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 24 17:08:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:15.217239 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a314be1-1feb-4fb3-8509-c52d4ba476df None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:15.229054 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dced598-2380-460a-be33-beb8f649edc3 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:15.265073 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e6e97d-3449-4a04-aded-a5627ffd13f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:15.265877 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e6e97d-3449-4a04-aded-a5627ffd13f2 None admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:15.265991 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e6e97d-3449-4a04-aded-a5627ffd13f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:15.266250 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e6e97d-3449-4a04-aded-a5627ffd13f2 None admin] PATH_INFO: `/v3/projects/46ab158bc3644c888557781b0630b0b8/users/557aca9255cb4bbb8af52e796c5c5177/roles/e7db2a847f56438bac13d0ff21c96b73` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:15.288043 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e6e97d-3449-4a04-aded-a5627ffd13f2 None admin] RBAC: Authorizing `identity:create_grant(project_id=46ab158bc3644c888557781b0630b0b8, user_id=557aca9255cb4bbb8af52e796c5c5177, role_id=e7db2a847f56438bac13d0ff21c96b73)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:15.289826 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e6e97d-3449-4a04-aded-a5627ffd13f2 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:15.299064 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:15.323254 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a314be1-1feb-4fb3-8509-c52d4ba476df None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:15.355518 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba182da-c15c-4d2e-80b0-6247c8a228fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:15.356607 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba182da-c15c-4d2e-80b0-6247c8a228fe None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:15.356891 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba182da-c15c-4d2e-80b0-6247c8a228fe None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:15.357158 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba182da-c15c-4d2e-80b0-6247c8a228fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:15.369436 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 295/592] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Aug 24 17:08:15 2022] PUT /identity/v3/projects/46ab158bc3644c888557781b0630b0b8/users/557aca9255cb4bbb8af52e796c5c5177/roles/e7db2a847f56438bac13d0ff21c96b73 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:08:15.369436 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e6e97d-3449-4a04-aded-a5627ffd13f2 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:15.369761 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba182da-c15c-4d2e-80b0-6247c8a228fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:15.370384 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba182da-c15c-4d2e-80b0-6247c8a228fe None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:15.387610 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-7ba182da-c15c-4d2e-80b0-6247c8a228fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:08:15.387610 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:15.387610 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:15.387610 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:15.387610 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:15.387610 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:15.387610 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:15.387610 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:15.387610 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:15.387610 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:15.387610 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:15.387610 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:15.387610 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:08:15.387610 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:08:15.387610 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:08:15.387610 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:08:15.388707 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:15.388707 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:15.388707 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:15.388707 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:15.388707 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:15.388707 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:15.388707 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:15.388707 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:15.388707 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:15.388707 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:15.388707 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:15.388707 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:15.388707 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:15.388707 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:15.388707 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:15.388707 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:15.388707 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:08:15.389610 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:08:15.389610 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:08:15.389610 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:08:15.393783 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 298/593] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 24 17:08:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:15.394795 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba182da-c15c-4d2e-80b0-6247c8a228fe None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:15.565609 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e6e97d-3449-4a04-aded-a5627ffd13f2 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:15.575983 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba182da-c15c-4d2e-80b0-6247c8a228fe None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:15.601629 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d98cf6-31e0-4cf6-a6f8-ec159d476d3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:15.602238 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d98cf6-31e0-4cf6-a6f8-ec159d476d3f None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:15.602513 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d98cf6-31e0-4cf6-a6f8-ec159d476d3f None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:15.602784 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d98cf6-31e0-4cf6-a6f8-ec159d476d3f None admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:15.603547 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d98cf6-31e0-4cf6-a6f8-ec159d476d3f None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:15.603860 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d98cf6-31e0-4cf6-a6f8-ec159d476d3f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:15.605268 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d98cf6-31e0-4cf6-a6f8-ec159d476d3f None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:15.609277 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:15.613233 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:15.624618 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74445bc9-d76e-43d7-bf3a-995dcb017949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:15.625371 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74445bc9-d76e-43d7-bf3a-995dcb017949 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:15.626001 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74445bc9-d76e-43d7-bf3a-995dcb017949 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:15.626368 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74445bc9-d76e-43d7-bf3a-995dcb017949 None admin] PATH_INFO: `/v3/domains` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:15.627144 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74445bc9-d76e-43d7-bf3a-995dcb017949 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:15.627543 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74445bc9-d76e-43d7-bf3a-995dcb017949 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:15.628998 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74445bc9-d76e-43d7-bf3a-995dcb017949 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:15.630490 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 296/594] 10.222.0.18 () {56 vars in 1137 bytes} [Wed Aug 24 17:08:15 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:15.631423 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d98cf6-31e0-4cf6-a6f8-ec159d476d3f None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:15.646486 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 299/595] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 24 17:08:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:15.647501 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74445bc9-d76e-43d7-bf3a-995dcb017949 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:15.746663 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d98cf6-31e0-4cf6-a6f8-ec159d476d3f None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:15.784118 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74445bc9-d76e-43d7-bf3a-995dcb017949 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:15.785152 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab14bb2-2ffc-4603-8a8f-0f9b4ca7b831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:15.785811 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab14bb2-2ffc-4603-8a8f-0f9b4ca7b831 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:15.786096 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab14bb2-2ffc-4603-8a8f-0f9b4ca7b831 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:15.786367 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab14bb2-2ffc-4603-8a8f-0f9b4ca7b831 None admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:15.787039 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab14bb2-2ffc-4603-8a8f-0f9b4ca7b831 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:15.787346 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab14bb2-2ffc-4603-8a8f-0f9b4ca7b831 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:15.788705 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab14bb2-2ffc-4603-8a8f-0f9b4ca7b831 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:15.800806 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 297/596] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 24 17:08:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:15.801787 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab14bb2-2ffc-4603-8a8f-0f9b4ca7b831 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:15.831925 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a782d0f-f263-4221-aa66-3cb54b0eaa18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:15.832977 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a782d0f-f263-4221-aa66-3cb54b0eaa18 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:15.833257 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a782d0f-f263-4221-aa66-3cb54b0eaa18 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:15.833529 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a782d0f-f263-4221-aa66-3cb54b0eaa18 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:15.848054 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a782d0f-f263-4221-aa66-3cb54b0eaa18 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:15.850002 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a782d0f-f263-4221-aa66-3cb54b0eaa18 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:15.863791 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:15.871178 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-6a782d0f-f263-4221-aa66-3cb54b0eaa18 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:08:15.871178 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:15.871178 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:15.871178 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:15.871178 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:15.871178 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:15.871178 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:15.871178 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:15.871178 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:15.871178 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:15.871178 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:15.871178 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:15.871178 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:08:15.871178 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:08:15.871178 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:08:15.871178 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:08:15.872706 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:15.872706 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:15.872706 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:15.872706 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:15.872706 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:15.872706 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:15.872706 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:15.872706 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:15.872706 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:15.872706 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:15.872706 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:15.872706 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:15.872706 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:15.872706 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:15.872706 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:15.872706 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:15.872706 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:08:15.873589 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:08:15.873589 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:08:15.873589 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:08:15.873589 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 300/597] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 24 17:08:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:15.874588 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a782d0f-f263-4221-aa66-3cb54b0eaa18 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:15.924411 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:15.936009 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab14bb2-2ffc-4603-8a8f-0f9b4ca7b831 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:15.985225 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece02f4c-164c-4ed8-a9e8-bd55aade58f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:15.985463 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a782d0f-f263-4221-aa66-3cb54b0eaa18 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:15.986295 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece02f4c-164c-4ed8-a9e8-bd55aade58f1 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:15.986991 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece02f4c-164c-4ed8-a9e8-bd55aade58f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:15.987412 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece02f4c-164c-4ed8-a9e8-bd55aade58f1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:15.996621 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece02f4c-164c-4ed8-a9e8-bd55aade58f1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:15.999345 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece02f4c-164c-4ed8-a9e8-bd55aade58f1 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:16.015200 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-ece02f4c-164c-4ed8-a9e8-bd55aade58f1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:08:16.015200 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:16.015200 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:16.015200 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:16.015200 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:16.015200 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:16.015200 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:16.015200 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:16.015200 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:16.015200 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:16.015200 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:16.015200 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:16.015200 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:08:16.015200 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:08:16.015200 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:08:16.016268 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:08:16.016268 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:16.016268 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:16.016268 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:16.016268 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:16.016268 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:16.016268 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:16.016268 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:16.016268 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:16.016268 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:16.016268 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:16.016268 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:16.016268 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:16.016268 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:16.016268 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:16.016268 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:16.016268 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:16.017416 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:08:16.017416 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:08:16.017416 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:08:16.017416 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:08:16.017416 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:08:16.017416 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:08:16.017416 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:08:16.017416 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:08:16.020221 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 298/598] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 24 17:08:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:16.021219 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece02f4c-164c-4ed8-a9e8-bd55aade58f1 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:16.031451 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e4cb21-377e-40a1-ab73-e658863adcf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:16.032190 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e4cb21-377e-40a1-ab73-e658863adcf5 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:16.032723 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e4cb21-377e-40a1-ab73-e658863adcf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:16.033073 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e4cb21-377e-40a1-ab73-e658863adcf5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:16.040952 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e4cb21-377e-40a1-ab73-e658863adcf5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:16.042815 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e4cb21-377e-40a1-ab73-e658863adcf5 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:16.056171 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-00e4cb21-377e-40a1-ab73-e658863adcf5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:08:16.056171 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:16.056171 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:16.056171 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:16.056171 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:16.056171 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:16.056171 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:16.056171 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:16.056171 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:16.056171 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:16.056171 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:16.056171 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:16.056171 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:08:16.056171 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:08:16.056171 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:08:16.057048 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:08:16.057048 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:16.057048 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:16.057048 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:16.057048 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:16.057048 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:16.057048 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:16.057048 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:16.057048 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:16.057048 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:16.057048 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:16.057048 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:16.057048 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:16.057048 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:16.057048 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:16.057048 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:16.057048 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:16.057918 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:08:16.057918 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:08:16.057918 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:08:16.057918 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:08:16.057918 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:08:16.057918 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:08:16.057918 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:08:16.057918 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:08:16.057918 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 301/599] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 24 17:08:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:16.058390 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e4cb21-377e-40a1-ab73-e658863adcf5 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:16.148861 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece02f4c-164c-4ed8-a9e8-bd55aade58f1 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:16.163731 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:16.181257 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:16.185886 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a3c69a-d3c9-455f-a51a-1c3eb1ad08fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:16.186231 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e4cb21-377e-40a1-ab73-e658863adcf5 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:16.186713 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a3c69a-d3c9-455f-a51a-1c3eb1ad08fd None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:16.187118 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a3c69a-d3c9-455f-a51a-1c3eb1ad08fd None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:16.187502 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a3c69a-d3c9-455f-a51a-1c3eb1ad08fd None admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:16.188486 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a3c69a-d3c9-455f-a51a-1c3eb1ad08fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:16.189260 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a3c69a-d3c9-455f-a51a-1c3eb1ad08fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:16.190883 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a3c69a-d3c9-455f-a51a-1c3eb1ad08fd None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:16.210049 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 299/600] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 24 17:08:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:16.210049 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a3c69a-d3c9-455f-a51a-1c3eb1ad08fd None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:16.236307 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed764c6b-ef5f-4635-8170-ce494ba07dd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:16.237144 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed764c6b-ef5f-4635-8170-ce494ba07dd6 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:16.237837 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed764c6b-ef5f-4635-8170-ce494ba07dd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:16.238254 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed764c6b-ef5f-4635-8170-ce494ba07dd6 None admin] PATH_INFO: `/v3/projects` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:16.239520 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed764c6b-ef5f-4635-8170-ce494ba07dd6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:16.240210 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed764c6b-ef5f-4635-8170-ce494ba07dd6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:16.242391 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed764c6b-ef5f-4635-8170-ce494ba07dd6 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:16.263701 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 302/601] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Aug 24 17:08:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:16.263701 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed764c6b-ef5f-4635-8170-ce494ba07dd6 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:16.337994 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a3c69a-d3c9-455f-a51a-1c3eb1ad08fd None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:16.375028 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed764c6b-ef5f-4635-8170-ce494ba07dd6 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:16.384530 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdf9291-0462-4e67-a39e-ba813bc75ba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:16.385376 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdf9291-0462-4e67-a39e-ba813bc75ba2 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:16.385826 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdf9291-0462-4e67-a39e-ba813bc75ba2 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:16.386702 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdf9291-0462-4e67-a39e-ba813bc75ba2 None admin] PATH_INFO: `/v3/projects` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:16.387610 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdf9291-0462-4e67-a39e-ba813bc75ba2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:16.388467 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdf9291-0462-4e67-a39e-ba813bc75ba2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:16.390396 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdf9291-0462-4e67-a39e-ba813bc75ba2 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:16.406962 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 300/602] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Aug 24 17:08:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:16.406962 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdf9291-0462-4e67-a39e-ba813bc75ba2 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:16.428149 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e20534b-e01e-4acc-8fa9-c6e9d9372bdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:16.428795 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e20534b-e01e-4acc-8fa9-c6e9d9372bdf None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:16.429079 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e20534b-e01e-4acc-8fa9-c6e9d9372bdf None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:16.429357 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e20534b-e01e-4acc-8fa9-c6e9d9372bdf None admin] PATH_INFO: `/v3/projects/service` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:16.439260 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e20534b-e01e-4acc-8fa9-c6e9d9372bdf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:16.441248 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e20534b-e01e-4acc-8fa9-c6e9d9372bdf None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:16.445360 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:16.455823 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-6e20534b-e01e-4acc-8fa9-c6e9d9372bdf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:08:16.455823 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:16.455823 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:16.455823 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:16.455823 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:16.455823 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:16.455823 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:16.455823 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:16.455823 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:16.455823 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:16.455823 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:16.455823 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:16.455823 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:08:16.455823 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:08:16.455823 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:08:16.462469 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:08:16.462469 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:16.462469 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:16.462469 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:16.462469 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:16.462469 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:16.462469 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:16.462469 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:16.462469 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:16.462469 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:16.462469 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:16.462469 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:16.462469 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:16.462469 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:16.462469 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:16.462469 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:16.462469 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:16.463385 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:08:16.463385 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:08:16.463385 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:08:16.463385 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:08:16.463385 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:08:16.463385 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:08:16.463385 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:08:16.463385 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:08:16.463385 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 303/603] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 24 17:08:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:16.463385 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e20534b-e01e-4acc-8fa9-c6e9d9372bdf None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:16.484048 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:16.534339 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdf9291-0462-4e67-a39e-ba813bc75ba2 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:16.585863 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7445c2-bc58-4412-91f9-1e61186cf8e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:16.586518 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7445c2-bc58-4412-91f9-1e61186cf8e2 None admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:16.586792 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7445c2-bc58-4412-91f9-1e61186cf8e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:16.587084 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7445c2-bc58-4412-91f9-1e61186cf8e2 None admin] PATH_INFO: `/v3/projects/46ab158bc3644c888557781b0630b0b8/users/3553e522363046f6828fd61b2a0642ed/roles/e7db2a847f56438bac13d0ff21c96b73` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:16.590799 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e20534b-e01e-4acc-8fa9-c6e9d9372bdf None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:16.606765 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7445c2-bc58-4412-91f9-1e61186cf8e2 None admin] RBAC: Authorizing `identity:create_grant(project_id=46ab158bc3644c888557781b0630b0b8, user_id=3553e522363046f6828fd61b2a0642ed, role_id=e7db2a847f56438bac13d0ff21c96b73)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:16.608614 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7445c2-bc58-4412-91f9-1e61186cf8e2 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:16.627256 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c20d271-4a44-4e93-b6b7-418b231a79c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:16.627907 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c20d271-4a44-4e93-b6b7-418b231a79c5 None admin] REQUEST_METHOD: `PUT` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:16.628183 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c20d271-4a44-4e93-b6b7-418b231a79c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:16.628450 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c20d271-4a44-4e93-b6b7-418b231a79c5 None admin] PATH_INFO: `/v3/projects/46ab158bc3644c888557781b0630b0b8/users/0e9ed4d47aef4140bfb2d3bb798cc559/roles/e7db2a847f56438bac13d0ff21c96b73` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:16.643107 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c20d271-4a44-4e93-b6b7-418b231a79c5 None admin] RBAC: Authorizing `identity:create_grant(project_id=46ab158bc3644c888557781b0630b0b8, user_id=0e9ed4d47aef4140bfb2d3bb798cc559, role_id=e7db2a847f56438bac13d0ff21c96b73)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:16.644514 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c20d271-4a44-4e93-b6b7-418b231a79c5 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:16.680223 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 301/604] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Aug 24 17:08:16 2022] PUT /identity/v3/projects/46ab158bc3644c888557781b0630b0b8/users/3553e522363046f6828fd61b2a0642ed/roles/e7db2a847f56438bac13d0ff21c96b73 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:08:16.681035 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7445c2-bc58-4412-91f9-1e61186cf8e2 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:16.748491 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 304/605] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Aug 24 17:08:16 2022] PUT /identity/v3/projects/46ab158bc3644c888557781b0630b0b8/users/0e9ed4d47aef4140bfb2d3bb798cc559/roles/e7db2a847f56438bac13d0ff21c96b73 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:08:16.751103 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb837b1e-3547-4b79-94cc-34d76fc505d0 None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:16.751917 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb837b1e-3547-4b79-94cc-34d76fc505d0 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:16.752395 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb837b1e-3547-4b79-94cc-34d76fc505d0 None None] PATH_INFO: `/` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:16.761288 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 305/606] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:08:16 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:08:16.765754 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feeef22-8b56-417a-ae45-4064e35c6740 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:16.766102 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feeef22-8b56-417a-ae45-4064e35c6740 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:16.766306 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feeef22-8b56-417a-ae45-4064e35c6740 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:16.795827 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-4feeef22-8b56-417a-ae45-4064e35c6740 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:08:16.797063 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-4feeef22-8b56-417a-ae45-4064e35c6740 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:08:16.838273 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:16.861585 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7445c2-bc58-4412-91f9-1e61186cf8e2 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:16.902065 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15382006-df6f-4ce2-a96e-f575cddb02c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:16.902691 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15382006-df6f-4ce2-a96e-f575cddb02c3 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:16.903143 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15382006-df6f-4ce2-a96e-f575cddb02c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:16.903421 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15382006-df6f-4ce2-a96e-f575cddb02c3 None admin] PATH_INFO: `/v3/projects` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:16.904128 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15382006-df6f-4ce2-a96e-f575cddb02c3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:16.904562 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15382006-df6f-4ce2-a96e-f575cddb02c3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:16.906003 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15382006-df6f-4ce2-a96e-f575cddb02c3 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:16.917517 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:16.923736 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 302/607] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Aug 24 17:08:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:16.930223 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15382006-df6f-4ce2-a96e-f575cddb02c3 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:16.931083 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 306/608] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:08:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 17:08:17.040082 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15382006-df6f-4ce2-a96e-f575cddb02c3 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:17.078179 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad06350-578d-4f6c-accd-25626ad4b7e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:17.079267 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad06350-578d-4f6c-accd-25626ad4b7e7 None admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:17.079762 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad06350-578d-4f6c-accd-25626ad4b7e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:17.080231 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad06350-578d-4f6c-accd-25626ad4b7e7 None admin] PATH_INFO: `/v3/projects/46ab158bc3644c888557781b0630b0b8/users/be06246d47584b529150ceb98a5cfd95/roles/e7db2a847f56438bac13d0ff21c96b73` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:17.097524 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad06350-578d-4f6c-accd-25626ad4b7e7 None admin] RBAC: Authorizing `identity:create_grant(project_id=46ab158bc3644c888557781b0630b0b8, user_id=be06246d47584b529150ceb98a5cfd95, role_id=e7db2a847f56438bac13d0ff21c96b73)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:17.099323 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad06350-578d-4f6c-accd-25626ad4b7e7 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:17.101876 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:17.196478 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 303/609] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Aug 24 17:08:16 2022] PUT /identity/v3/projects/46ab158bc3644c888557781b0630b0b8/users/be06246d47584b529150ceb98a5cfd95/roles/e7db2a847f56438bac13d0ff21c96b73 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:08:17.229366 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4feeef22-8b56-417a-ae45-4064e35c6740 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:17.367034 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4feeef22-8b56-417a-ae45-4064e35c6740 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:17.404817 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeef604f-e874-49ea-ad8c-6dda3e9b7981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:17.405871 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeef604f-e874-49ea-ad8c-6dda3e9b7981 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:17.406375 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeef604f-e874-49ea-ad8c-6dda3e9b7981 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:17.406839 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeef604f-e874-49ea-ad8c-6dda3e9b7981 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:17.419352 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeef604f-e874-49ea-ad8c-6dda3e9b7981 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:08:17.424619 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-eeef604f-e874-49ea-ad8c-6dda3e9b7981 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:08:17.424619 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:17.424619 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:17.424619 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:17.424619 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:17.424619 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:17.424619 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:17.424619 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:17.424619 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:17.424619 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:17.424619 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:17.424619 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:17.424619 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:08:17.424619 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:08:17.424619 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:08:17.424619 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:08:17.425640 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:08:17.425640 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:08:17.425640 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:08:17.425640 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:08:17.425640 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:17.425640 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:17.425640 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:17.425640 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:17.425640 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:17.425640 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:17.425640 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:17.425640 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:17.425640 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:17.425640 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:17.425640 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:17.425640 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:17.426463 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:17.426463 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:17.426463 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:17.426463 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:17.426463 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:08:17.426463 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:08:17.426463 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:08:17.426463 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:08:17.426463 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:08:17.426463 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:08:17.426463 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:08:17.426463 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:08:17.430282 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 307/610] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Aug 24 17:08:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:17.435911 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad06350-578d-4f6c-accd-25626ad4b7e7 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:17.452409 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941214e8-1567-4c11-8eb1-149fb3758c27 None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:17.453210 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941214e8-1567-4c11-8eb1-149fb3758c27 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:17.453682 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941214e8-1567-4c11-8eb1-149fb3758c27 None None] PATH_INFO: `/` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:17.455692 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 308/611] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:08:17 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:08:17.464265 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabb59dc-1f1b-4e00-98e9-3d03a0805f1f None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:17.465002 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabb59dc-1f1b-4e00-98e9-3d03a0805f1f None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:17.466209 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabb59dc-1f1b-4e00-98e9-3d03a0805f1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:17.490057 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-cabb59dc-1f1b-4e00-98e9-3d03a0805f1f None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:08:17.491650 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-cabb59dc-1f1b-4e00-98e9-3d03a0805f1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:08:17.567895 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:17.582139 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 309/612] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:08:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 17:08:17.609814 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad06350-578d-4f6c-accd-25626ad4b7e7 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:17.651306 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3290c25-e791-41f8-a699-7ca847fbaf9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:17.652297 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3290c25-e791-41f8-a699-7ca847fbaf9f None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:17.652783 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3290c25-e791-41f8-a699-7ca847fbaf9f None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:17.653812 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3290c25-e791-41f8-a699-7ca847fbaf9f None admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:17.654683 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3290c25-e791-41f8-a699-7ca847fbaf9f None admin] RBAC: Adding query filter params (name=service) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:17.655211 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3290c25-e791-41f8-a699-7ca847fbaf9f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:17.656809 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3290c25-e791-41f8-a699-7ca847fbaf9f None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:17.671430 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 304/613] 10.222.0.18 () {56 vars in 1103 bytes} [Wed Aug 24 17:08:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:17.676452 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabb59dc-1f1b-4e00-98e9-3d03a0805f1f None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:17.800771 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabb59dc-1f1b-4e00-98e9-3d03a0805f1f None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:17.847411 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbaaa759-d9d4-471f-ad84-f3c08181abdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:17.848443 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaaa759-d9d4-471f-ad84-f3c08181abdd None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:17.848931 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaaa759-d9d4-471f-ad84-f3c08181abdd None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:17.849406 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaaa759-d9d4-471f-ad84-f3c08181abdd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:17.866213 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbaaa759-d9d4-471f-ad84-f3c08181abdd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:17.867978 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbaaa759-d9d4-471f-ad84-f3c08181abdd None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:17.886729 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-bbaaa759-d9d4-471f-ad84-f3c08181abdd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:08:17.886729 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:17.886729 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:17.886729 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:17.886729 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:17.886729 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:17.886729 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:17.886729 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:17.886729 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:17.886729 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:17.886729 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:17.886729 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:17.886729 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:08:17.886729 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:08:17.886729 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:08:17.886729 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:08:17.888283 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:17.888283 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:17.888283 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:17.888283 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:17.888283 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:17.888283 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:17.888283 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:17.888283 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:17.888283 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:17.888283 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:17.888283 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:17.888283 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:17.888283 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:17.888283 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:17.888283 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:17.888283 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:17.888283 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:08:17.889196 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:08:17.889196 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:08:17.889196 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:08:17.890956 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 310/614] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 24 17:08:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:17.896514 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3290c25-e791-41f8-a699-7ca847fbaf9f None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:17.978140 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:18.023089 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3290c25-e791-41f8-a699-7ca847fbaf9f None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:18.071691 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8b266e-865e-4144-bfa1-c53961b66ab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:18.072307 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8b266e-865e-4144-bfa1-c53961b66ab8 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:18.072590 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8b266e-865e-4144-bfa1-c53961b66ab8 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:18.072858 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8b266e-865e-4144-bfa1-c53961b66ab8 None admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:18.073508 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8b266e-865e-4144-bfa1-c53961b66ab8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:18.073807 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8b266e-865e-4144-bfa1-c53961b66ab8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:18.075824 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8b266e-865e-4144-bfa1-c53961b66ab8 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:18.094661 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 305/615] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 24 17:08:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:18.099996 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbaaa759-d9d4-471f-ad84-f3c08181abdd None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:18.108490 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8b266e-865e-4144-bfa1-c53961b66ab8 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:18.185792 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:18.239693 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8b266e-865e-4144-bfa1-c53961b66ab8 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:18.276328 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:18.307858 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edff387e-1c24-4047-a807-3c7546ec7863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:18.308503 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edff387e-1c24-4047-a807-3c7546ec7863 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:18.308785 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edff387e-1c24-4047-a807-3c7546ec7863 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:18.309061 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edff387e-1c24-4047-a807-3c7546ec7863 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:18.316287 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbaaa759-d9d4-471f-ad84-f3c08181abdd None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:18.333649 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edff387e-1c24-4047-a807-3c7546ec7863 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:08:18.338196 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-edff387e-1c24-4047-a807-3c7546ec7863 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:08:18.338196 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:18.338196 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:18.338196 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:18.338196 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:18.338196 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:18.338196 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:18.338196 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:18.338196 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:18.338196 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:18.338196 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:18.338196 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:18.338196 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:08:18.338196 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:08:18.338196 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:08:18.338196 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:08:18.339123 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:08:18.339123 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:08:18.339123 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:08:18.339123 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:08:18.339123 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:18.339123 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:18.339123 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:18.339123 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:18.339123 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:18.339123 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:18.339123 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:18.339123 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:18.339123 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:18.339123 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:18.339123 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:18.339123 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:18.340426 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:18.340426 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:18.340426 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:18.340426 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:18.340426 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:08:18.340426 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:08:18.340426 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:08:18.340426 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:08:18.340426 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:08:18.340426 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:08:18.340426 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:08:18.340426 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:08:18.342187 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 306/616] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Aug 24 17:08:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:08:18.347174 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edff387e-1c24-4047-a807-3c7546ec7863 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:18.378023 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbce10b3-e5fe-4fab-9920-6029bac7a11f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:18.378656 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbce10b3-e5fe-4fab-9920-6029bac7a11f None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:18.378955 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbce10b3-e5fe-4fab-9920-6029bac7a11f None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:18.379235 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbce10b3-e5fe-4fab-9920-6029bac7a11f None admin] PATH_INFO: `/v3/users/swift` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:18.391946 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbce10b3-e5fe-4fab-9920-6029bac7a11f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:18.393391 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbce10b3-e5fe-4fab-9920-6029bac7a11f None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:18.412984 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-cbce10b3-e5fe-4fab-9920-6029bac7a11f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 17:08:18.412984 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:18.412984 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:18.412984 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:18.412984 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:18.412984 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:18.412984 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:18.412984 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:18.412984 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:18.412984 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:18.412984 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:18.412984 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:18.412984 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:08:18.412984 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:08:18.412984 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:08:18.412984 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:08:18.413932 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:18.413932 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:18.413932 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:08:18.413932 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:08:18.413932 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:08:18.413932 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:08:18.413932 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:18.413932 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:18.413932 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:18.413932 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:18.413932 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:18.413932 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:18.413932 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:18.413932 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:18.413932 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:18.413932 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:18.413932 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:18.413932 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:18.415520 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:18.415520 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:18.415520 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:08:18.415520 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:08:18.415520 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:08:18.415520 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:08:18.415520 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:08:18.415520 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:08:18.415520 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 17:08:18.415520 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:08:18.415520 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 311/617] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Aug 24 17:08:18 2022] GET /identity/v3/users/swift => generated 83 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:18.424156 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbce10b3-e5fe-4fab-9920-6029bac7a11f None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:18.542518 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:18.545957 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbce10b3-e5fe-4fab-9920-6029bac7a11f None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:18.563032 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edff387e-1c24-4047-a807-3c7546ec7863 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:18.600717 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d545cf9-3396-40aa-ae52-ad8dfe5c6071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:18.601243 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d545cf9-3396-40aa-ae52-ad8dfe5c6071 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:18.602133 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d545cf9-3396-40aa-ae52-ad8dfe5c6071 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:18.602410 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d545cf9-3396-40aa-ae52-ad8dfe5c6071 None admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:18.603115 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d545cf9-3396-40aa-ae52-ad8dfe5c6071 None admin] RBAC: Adding query filter params (name=service) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:18.603424 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d545cf9-3396-40aa-ae52-ad8dfe5c6071 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:18.604823 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d545cf9-3396-40aa-ae52-ad8dfe5c6071 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:18.611242 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1d5a2d-b721-4f10-92d5-e4afc5759253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:18.611873 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1d5a2d-b721-4f10-92d5-e4afc5759253 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:18.612158 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1d5a2d-b721-4f10-92d5-e4afc5759253 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:18.612432 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1d5a2d-b721-4f10-92d5-e4afc5759253 None admin] PATH_INFO: `/v3/users` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:18.613196 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1d5a2d-b721-4f10-92d5-e4afc5759253 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:18.613506 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1d5a2d-b721-4f10-92d5-e4afc5759253 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:18.615028 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1d5a2d-b721-4f10-92d5-e4afc5759253 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:18.630114 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 307/618] 10.222.0.18 () {56 vars in 1103 bytes} [Wed Aug 24 17:08:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:18.635690 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d545cf9-3396-40aa-ae52-ad8dfe5c6071 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:18.655685 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 312/619] 10.222.0.18 () {56 vars in 1135 bytes} [Wed Aug 24 17:08:18 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:08:18.661007 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1d5a2d-b721-4f10-92d5-e4afc5759253 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:18.719400 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:18.842609 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1d5a2d-b721-4f10-92d5-e4afc5759253 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:18.846895 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d545cf9-3396-40aa-ae52-ad8dfe5c6071 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:18.906013 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c16366a-a83f-4c9c-b01e-fd4716111ebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:18.906737 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c16366a-a83f-4c9c-b01e-fd4716111ebb None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:18.906862 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c16366a-a83f-4c9c-b01e-fd4716111ebb None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:18.907108 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c16366a-a83f-4c9c-b01e-fd4716111ebb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:18.918963 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb2b856-1d32-4351-8bfa-0d1795bbf8e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:18.919602 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb2b856-1d32-4351-8bfa-0d1795bbf8e0 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:18.919876 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb2b856-1d32-4351-8bfa-0d1795bbf8e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:18.920141 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb2b856-1d32-4351-8bfa-0d1795bbf8e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:18.926935 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c16366a-a83f-4c9c-b01e-fd4716111ebb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:18.928393 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c16366a-a83f-4c9c-b01e-fd4716111ebb None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:18.944836 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb2b856-1d32-4351-8bfa-0d1795bbf8e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:18.946282 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb2b856-1d32-4351-8bfa-0d1795bbf8e0 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:18.959793 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-2c16366a-a83f-4c9c-b01e-fd4716111ebb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:08:18.959793 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:18.959793 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:18.959793 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:18.959793 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:18.959793 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:18.959793 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:18.959793 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:18.959793 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:18.959793 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:18.959793 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:18.959793 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:18.959793 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:08:18.959793 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:08:18.959793 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:08:18.959793 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:08:18.960785 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:18.960785 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:18.960785 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:18.960785 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:18.960785 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:18.960785 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:18.960785 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:18.960785 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:18.960785 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:18.960785 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:18.960785 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:18.960785 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:18.960785 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:18.960785 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:18.960785 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:18.960785 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:18.960785 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:08:18.961681 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:08:18.961681 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:08:18.961681 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:08:18.961681 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 313/620] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 24 17:08:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:18.971841 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c16366a-a83f-4c9c-b01e-fd4716111ebb None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:18.973411 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-deb2b856-1d32-4351-8bfa-0d1795bbf8e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:08:18.973411 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:18.973411 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:18.973411 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:18.973411 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:18.973411 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:18.973411 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:18.973411 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:18.973411 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:18.973411 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:18.973411 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:18.973411 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:18.973411 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:08:18.973411 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:08:18.973411 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:08:18.973411 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:08:18.975536 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:18.975536 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:18.975536 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:18.975536 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:18.975536 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:18.975536 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:18.975536 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:18.975536 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:18.975536 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:18.975536 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:18.975536 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:18.975536 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:18.975536 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:18.975536 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:18.975536 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:18.975536 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:18.975536 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:08:18.977607 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:08:18.977607 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:08:18.977607 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:08:18.977607 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 308/621] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 24 17:08:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 339 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:18.980349 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb2b856-1d32-4351-8bfa-0d1795bbf8e0 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:18.991420 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:19.106650 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb2b856-1d32-4351-8bfa-0d1795bbf8e0 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:19.114789 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c16366a-a83f-4c9c-b01e-fd4716111ebb None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:19.131534 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:19.150128 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebcc1e6-05f7-494b-add6-38cd94021f9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:19.152231 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebcc1e6-05f7-494b-add6-38cd94021f9d None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:19.152519 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebcc1e6-05f7-494b-add6-38cd94021f9d None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:19.154385 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebcc1e6-05f7-494b-add6-38cd94021f9d None admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:19.154385 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebcc1e6-05f7-494b-add6-38cd94021f9d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:19.154385 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebcc1e6-05f7-494b-add6-38cd94021f9d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:19.155352 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e8e6e4-15fd-41f3-a811-5ecf82eaa5b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:19.156253 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e8e6e4-15fd-41f3-a811-5ecf82eaa5b8 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:19.156733 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e8e6e4-15fd-41f3-a811-5ecf82eaa5b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:19.157198 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e8e6e4-15fd-41f3-a811-5ecf82eaa5b8 None admin] PATH_INFO: `/v3/domains` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:19.157780 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebcc1e6-05f7-494b-add6-38cd94021f9d None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:19.158307 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e8e6e4-15fd-41f3-a811-5ecf82eaa5b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:19.158819 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e8e6e4-15fd-41f3-a811-5ecf82eaa5b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:19.162583 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e8e6e4-15fd-41f3-a811-5ecf82eaa5b8 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:19.171533 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 309/622] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 24 17:08:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:08:19.173199 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f20050d-2d62-4421-9148-d807a0e6e5d1 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:19.173316 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f20050d-2d62-4421-9148-d807a0e6e5d1 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:19.173592 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f20050d-2d62-4421-9148-d807a0e6e5d1 None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:19.178752 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 310/623] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:08:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:08:19.178752 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f20050d-2d62-4421-9148-d807a0e6e5d1 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:19.181107 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 314/624] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 24 17:08:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:19.182603 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14e09e0-a3cd-4e94-a4ce-069ad0627c52 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:19.183271 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14e09e0-a3cd-4e94-a4ce-069ad0627c52 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:19.183749 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14e09e0-a3cd-4e94-a4ce-069ad0627c52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:19.202348 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-a14e09e0-a3cd-4e94-a4ce-069ad0627c52 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:08:19.203907 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-a14e09e0-a3cd-4e94-a4ce-069ad0627c52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:08:19.270418 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:19.275714 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 315/625] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:08:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:08:19.276746 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14e09e0-a3cd-4e94-a4ce-069ad0627c52 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:19.296797 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f20050d-2d62-4421-9148-d807a0e6e5d1 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:19.333988 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9beab19d-1d1a-4850-b294-d94bc2cf1c59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:19.334639 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beab19d-1d1a-4850-b294-d94bc2cf1c59 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:19.334927 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beab19d-1d1a-4850-b294-d94bc2cf1c59 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:19.335208 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beab19d-1d1a-4850-b294-d94bc2cf1c59 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:19.342800 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9beab19d-1d1a-4850-b294-d94bc2cf1c59 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:19.344515 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9beab19d-1d1a-4850-b294-d94bc2cf1c59 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:19.356666 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-9beab19d-1d1a-4850-b294-d94bc2cf1c59 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 17:08:19.356666 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:19.356666 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:19.356666 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:19.356666 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:19.356666 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:19.356666 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:19.356666 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:19.356666 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:19.356666 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:19.356666 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:19.356666 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:19.356666 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:08:19.356666 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:08:19.356666 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:08:19.356666 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:08:19.357639 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:19.357639 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:19.357639 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:08:19.357639 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:08:19.357639 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:08:19.357639 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:08:19.357639 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:19.357639 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:19.357639 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:19.357639 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:19.357639 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:19.357639 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:19.357639 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:19.357639 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:19.357639 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:19.357639 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:19.357639 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:19.357639 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:19.359002 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:19.359002 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:19.359002 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:08:19.359002 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:08:19.359002 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:08:19.359002 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:08:19.359002 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:08:19.359002 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:08:19.359002 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 17:08:19.359002 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:08:19.359002 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 311/626] 10.222.0.18 () {56 vars in 1093 bytes} [Wed Aug 24 17:08:19 2022] GET /identity/v3/users/nova => generated 82 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:19.359733 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9928878f-1952-4bf8-802b-5d591cd1bc69 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:19.359733 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9928878f-1952-4bf8-802b-5d591cd1bc69 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:19.359864 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9928878f-1952-4bf8-802b-5d591cd1bc69 None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:19.365214 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 312/627] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:08:19 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:08:19.365977 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9928878f-1952-4bf8-802b-5d591cd1bc69 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:19.379722 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:19.394368 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14e09e0-a3cd-4e94-a4ce-069ad0627c52 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:19.431157 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9abee0-0b25-413a-807c-ba4acf90f228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:19.432157 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9abee0-0b25-413a-807c-ba4acf90f228 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:19.433019 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9abee0-0b25-413a-807c-ba4acf90f228 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:19.433532 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9abee0-0b25-413a-807c-ba4acf90f228 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:19.441695 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9abee0-0b25-413a-807c-ba4acf90f228 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:19.444207 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9abee0-0b25-413a-807c-ba4acf90f228 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:19.458651 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-3d9abee0-0b25-413a-807c-ba4acf90f228 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:08:19.458651 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:19.458651 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:19.458651 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:19.458651 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:19.458651 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:19.458651 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:19.458651 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:19.458651 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:19.458651 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:19.458651 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:19.458651 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:19.458651 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:08:19.458651 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:08:19.458651 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:08:19.459825 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:08:19.459825 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:19.459825 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:19.459825 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:19.459825 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:19.459825 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:19.459825 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:19.459825 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:19.459825 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:19.459825 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:19.459825 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:19.459825 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:19.459825 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:19.459825 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:19.459825 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:19.459825 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:19.459825 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:19.460843 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:08:19.460843 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:08:19.460843 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:08:19.460843 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:08:19.460843 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:08:19.460843 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:08:19.460843 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:08:19.460843 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:08:19.462878 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 316/628] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 24 17:08:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:19.464634 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da05578d-40ce-404c-8963-0a6303810fba None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:19.465084 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da05578d-40ce-404c-8963-0a6303810fba None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:19.466083 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da05578d-40ce-404c-8963-0a6303810fba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:19.474958 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9928878f-1952-4bf8-802b-5d591cd1bc69 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:19.490237 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-da05578d-40ce-404c-8963-0a6303810fba None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:08:19.491761 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-da05578d-40ce-404c-8963-0a6303810fba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:08:19.511681 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a8932f-3b7e-462a-a861-3e1905882d1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:19.512441 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a8932f-3b7e-462a-a861-3e1905882d1f None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:19.512722 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a8932f-3b7e-462a-a861-3e1905882d1f None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:19.512985 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a8932f-3b7e-462a-a861-3e1905882d1f None admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:19.513711 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a8932f-3b7e-462a-a861-3e1905882d1f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:19.514014 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a8932f-3b7e-462a-a861-3e1905882d1f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:19.515466 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a8932f-3b7e-462a-a861-3e1905882d1f None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:19.535604 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 313/629] 10.222.0.18 () {56 vars in 1133 bytes} [Wed Aug 24 17:08:19 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:19.536453 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a8932f-3b7e-462a-a861-3e1905882d1f None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:19.560281 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:19.571768 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 317/630] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:08:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:08:19.572909 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da05578d-40ce-404c-8963-0a6303810fba None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:19.630978 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:19.668739 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a8932f-3b7e-462a-a861-3e1905882d1f None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:19.683879 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da05578d-40ce-404c-8963-0a6303810fba None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:19.705179 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c95707b-a827-47ae-ab77-24ff8473605f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:19.705792 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c95707b-a827-47ae-ab77-24ff8473605f None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:19.706070 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c95707b-a827-47ae-ab77-24ff8473605f None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:19.706343 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c95707b-a827-47ae-ab77-24ff8473605f None admin] PATH_INFO: `/v3/projects` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:19.707077 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c95707b-a827-47ae-ab77-24ff8473605f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:19.707385 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c95707b-a827-47ae-ab77-24ff8473605f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:19.708811 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c95707b-a827-47ae-ab77-24ff8473605f None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:19.721357 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 314/631] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Aug 24 17:08:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:19.722148 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c95707b-a827-47ae-ab77-24ff8473605f None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:19.727105 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76dc917-7616-4777-b6c5-b32ade65a896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:19.727723 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76dc917-7616-4777-b6c5-b32ade65a896 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:19.728000 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76dc917-7616-4777-b6c5-b32ade65a896 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:19.728267 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76dc917-7616-4777-b6c5-b32ade65a896 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:19.793555 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:19.804295 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76dc917-7616-4777-b6c5-b32ade65a896 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:19.805991 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76dc917-7616-4777-b6c5-b32ade65a896 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:19.839196 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c95707b-a827-47ae-ab77-24ff8473605f None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:19.865541 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:19.873112 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e10d57-b434-43d9-a2b9-ca06a43ca358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:19.873739 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e10d57-b434-43d9-a2b9-ca06a43ca358 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:19.886174 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e10d57-b434-43d9-a2b9-ca06a43ca358 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:19.886174 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e10d57-b434-43d9-a2b9-ca06a43ca358 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:19.886174 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e10d57-b434-43d9-a2b9-ca06a43ca358 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:08:19.890373 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-77e10d57-b434-43d9-a2b9-ca06a43ca358 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:08:19.890373 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:19.890373 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:19.890373 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:19.890373 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:19.890373 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:19.890373 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:19.890373 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:19.890373 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:19.890373 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:19.890373 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:19.890373 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:19.890373 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:08:19.890373 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:08:19.890373 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:08:19.890373 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:08:19.891374 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:08:19.891374 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:08:19.891374 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:08:19.891374 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:08:19.891374 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:19.891374 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:19.891374 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:19.891374 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:19.891374 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:19.891374 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:19.891374 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:19.891374 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:19.891374 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:19.891374 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:19.891374 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:19.891374 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:19.892460 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:19.892460 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:19.892460 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:19.892460 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:19.892460 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:08:19.892460 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:08:19.892460 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:08:19.892460 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:08:19.892460 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:08:19.892460 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:08:19.892460 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:08:19.892460 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:08:19.892460 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 315/632] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Aug 24 17:08:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:19.893493 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0195c85-5701-42f6-b830-1daecf210a5f None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:19.893493 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0195c85-5701-42f6-b830-1daecf210a5f None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:19.893493 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0195c85-5701-42f6-b830-1daecf210a5f None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:19.894642 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 316/633] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:08:19 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:08:19.899445 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0195c85-5701-42f6-b830-1daecf210a5f None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:19.910315 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 318/634] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 24 17:08:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 17:08:19.911178 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76dc917-7616-4777-b6c5-b32ade65a896 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:19.998757 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:19.999233 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0195c85-5701-42f6-b830-1daecf210a5f None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:20.008069 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76dc917-7616-4777-b6c5-b32ade65a896 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:20.033149 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d918cf-4e6f-4dfb-9360-f6ef23b091a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:20.034007 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d918cf-4e6f-4dfb-9360-f6ef23b091a8 None admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:20.034295 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d918cf-4e6f-4dfb-9360-f6ef23b091a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:20.034570 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d918cf-4e6f-4dfb-9360-f6ef23b091a8 None admin] PATH_INFO: `/v3/projects/46ab158bc3644c888557781b0630b0b8/users/ef8caa353be34499a58cd25b72ee4032/roles/e7db2a847f56438bac13d0ff21c96b73` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:20.042370 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28982693-1918-4522-a3fb-52b584c3058a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:20.043030 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28982693-1918-4522-a3fb-52b584c3058a None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:20.043313 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28982693-1918-4522-a3fb-52b584c3058a None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:20.043617 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28982693-1918-4522-a3fb-52b584c3058a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:20.048654 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d918cf-4e6f-4dfb-9360-f6ef23b091a8 None admin] RBAC: Authorizing `identity:create_grant(project_id=46ab158bc3644c888557781b0630b0b8, user_id=ef8caa353be34499a58cd25b72ee4032, role_id=e7db2a847f56438bac13d0ff21c96b73)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:20.050093 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d918cf-4e6f-4dfb-9360-f6ef23b091a8 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:20.053349 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28982693-1918-4522-a3fb-52b584c3058a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:08:20.057976 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-28982693-1918-4522-a3fb-52b584c3058a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:08:20.057976 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:20.057976 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:20.057976 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:20.057976 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:20.057976 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:20.057976 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:20.057976 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:20.057976 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:20.057976 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:20.057976 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:20.057976 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:20.057976 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:08:20.057976 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:08:20.057976 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:08:20.057976 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:08:20.063443 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:08:20.063443 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:08:20.063443 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:08:20.063443 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:08:20.063443 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:20.063443 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:20.063443 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:20.063443 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:20.063443 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:20.063443 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:20.063443 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:20.063443 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:20.063443 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:20.063443 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:20.063443 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:20.063443 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:20.064359 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:20.064359 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:20.064359 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:20.064359 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:20.064359 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:08:20.064359 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:08:20.064359 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:08:20.064359 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:08:20.064359 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:08:20.064359 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:08:20.064359 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:08:20.064359 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:08:20.064359 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 319/635] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Aug 24 17:08:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:20.064359 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28982693-1918-4522-a3fb-52b584c3058a None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:20.109843 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 317/636] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Aug 24 17:08:19 2022] PUT /identity/v3/projects/46ab158bc3644c888557781b0630b0b8/users/ef8caa353be34499a58cd25b72ee4032/roles/e7db2a847f56438bac13d0ff21c96b73 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:08:20.111354 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761ea142-fab6-466b-be5c-5e7746153ace None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:20.111634 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761ea142-fab6-466b-be5c-5e7746153ace None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:20.111908 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761ea142-fab6-466b-be5c-5e7746153ace None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:20.147348 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-761ea142-fab6-466b-be5c-5e7746153ace None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:08:20.149110 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-761ea142-fab6-466b-be5c-5e7746153ace None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:08:20.165073 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28982693-1918-4522-a3fb-52b584c3058a None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:20.186294 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:20.205544 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0944b1b-dddb-478b-99b7-5f464dd67e2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:20.206330 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0944b1b-dddb-478b-99b7-5f464dd67e2b None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:20.206470 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0944b1b-dddb-478b-99b7-5f464dd67e2b None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:20.206680 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0944b1b-dddb-478b-99b7-5f464dd67e2b None admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:20.207559 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0944b1b-dddb-478b-99b7-5f464dd67e2b None admin] RBAC: Adding query filter params (name=service) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:20.207667 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0944b1b-dddb-478b-99b7-5f464dd67e2b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:20.209087 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0944b1b-dddb-478b-99b7-5f464dd67e2b None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:20.223321 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 320/637] 10.222.0.18 () {56 vars in 1103 bytes} [Wed Aug 24 17:08:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:20.223321 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0944b1b-dddb-478b-99b7-5f464dd67e2b None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:20.235341 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 318/638] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:08:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:08:20.240917 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:20.247923 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761ea142-fab6-466b-be5c-5e7746153ace None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:20.343572 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0944b1b-dddb-478b-99b7-5f464dd67e2b None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:20.378410 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761ea142-fab6-466b-be5c-5e7746153ace None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:20.381162 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231ec980-957d-4b4e-82d6-e44b1db3a803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:20.382019 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231ec980-957d-4b4e-82d6-e44b1db3a803 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:20.382489 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231ec980-957d-4b4e-82d6-e44b1db3a803 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:20.383023 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231ec980-957d-4b4e-82d6-e44b1db3a803 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:20.396140 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231ec980-957d-4b4e-82d6-e44b1db3a803 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:20.397942 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231ec980-957d-4b4e-82d6-e44b1db3a803 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:20.414247 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-231ec980-957d-4b4e-82d6-e44b1db3a803 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:08:20.414247 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:20.414247 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:20.414247 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:20.414247 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:20.414247 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:20.414247 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:20.414247 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:20.414247 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:20.414247 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:20.414247 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:20.414247 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:20.414247 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:08:20.414247 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:08:20.414247 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:08:20.414247 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:08:20.415476 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:20.415476 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:20.415476 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:20.415476 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:20.415476 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:20.415476 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:20.415476 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:20.415476 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:20.415476 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:20.415476 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:20.415476 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:20.415476 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:20.415476 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:20.415476 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:20.415476 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:20.415476 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:20.415476 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:08:20.416581 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:08:20.416581 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:08:20.416581 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:08:20.418313 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 321/639] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 24 17:08:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:20.419280 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de87668-ad64-4b45-ba56-9bf3272ade67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:20.419518 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231ec980-957d-4b4e-82d6-e44b1db3a803 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:20.420013 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de87668-ad64-4b45-ba56-9bf3272ade67 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:20.420358 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de87668-ad64-4b45-ba56-9bf3272ade67 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:20.420715 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de87668-ad64-4b45-ba56-9bf3272ade67 None admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:20.421431 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de87668-ad64-4b45-ba56-9bf3272ade67 None admin] RBAC: Adding query filter params (name=service) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:20.421801 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de87668-ad64-4b45-ba56-9bf3272ade67 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:20.423662 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de87668-ad64-4b45-ba56-9bf3272ade67 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:20.438862 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 319/640] 10.222.0.18 () {56 vars in 1103 bytes} [Wed Aug 24 17:08:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:20.438862 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de87668-ad64-4b45-ba56-9bf3272ade67 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:20.473988 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:20.499864 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:20.542611 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231ec980-957d-4b4e-82d6-e44b1db3a803 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:20.559892 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de87668-ad64-4b45-ba56-9bf3272ade67 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:20.581038 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665f84f7-78df-4f1d-8946-0c7184e0d5f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:20.581672 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665f84f7-78df-4f1d-8946-0c7184e0d5f0 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:20.581938 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665f84f7-78df-4f1d-8946-0c7184e0d5f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:20.582202 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665f84f7-78df-4f1d-8946-0c7184e0d5f0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:20.593963 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665f84f7-78df-4f1d-8946-0c7184e0d5f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:20.595375 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665f84f7-78df-4f1d-8946-0c7184e0d5f0 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:20.596262 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d3154a-c588-44be-ae55-e1be4c014699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:20.596872 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d3154a-c588-44be-ae55-e1be4c014699 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:20.597220 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d3154a-c588-44be-ae55-e1be4c014699 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:20.597504 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d3154a-c588-44be-ae55-e1be4c014699 None admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:20.598160 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d3154a-c588-44be-ae55-e1be4c014699 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:20.598465 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d3154a-c588-44be-ae55-e1be4c014699 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:20.599847 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d3154a-c588-44be-ae55-e1be4c014699 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:20.610229 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-665f84f7-78df-4f1d-8946-0c7184e0d5f0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:08:20.610229 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:20.610229 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:20.610229 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:20.610229 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:20.610229 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:20.610229 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:20.610229 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:20.610229 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:20.610229 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:20.610229 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:20.610229 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:20.610229 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:08:20.610229 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:08:20.610229 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:08:20.610229 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:08:20.611249 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:20.611249 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:20.611249 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:20.611249 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:20.611249 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:20.611249 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:20.611249 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:20.611249 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:20.611249 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:20.611249 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:20.611249 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:20.611249 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:20.611249 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:20.611249 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:20.611249 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:20.611249 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:20.611249 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:08:20.612175 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:08:20.612175 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:08:20.612175 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:08:20.612175 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 322/641] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 24 17:08:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:20.612175 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665f84f7-78df-4f1d-8946-0c7184e0d5f0 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:20.619030 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 320/642] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 24 17:08:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:20.619030 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d3154a-c588-44be-ae55-e1be4c014699 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:20.717762 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:20.723976 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665f84f7-78df-4f1d-8946-0c7184e0d5f0 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:20.728351 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d3154a-c588-44be-ae55-e1be4c014699 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:20.744870 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:20.760985 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8647b448-2c81-4636-bb94-2943ad93573d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:20.761830 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8647b448-2c81-4636-bb94-2943ad93573d None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:20.762112 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8647b448-2c81-4636-bb94-2943ad93573d None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:20.763060 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8647b448-2c81-4636-bb94-2943ad93573d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:20.769949 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63eb2db7-33fc-4cbe-b1bb-0e76e2a03463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:20.770591 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63eb2db7-33fc-4cbe-b1bb-0e76e2a03463 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:20.770868 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63eb2db7-33fc-4cbe-b1bb-0e76e2a03463 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:20.771159 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63eb2db7-33fc-4cbe-b1bb-0e76e2a03463 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:20.775688 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8647b448-2c81-4636-bb94-2943ad93573d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:20.777345 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8647b448-2c81-4636-bb94-2943ad93573d None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:20.783817 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63eb2db7-33fc-4cbe-b1bb-0e76e2a03463 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:08:20.791816 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-63eb2db7-33fc-4cbe-b1bb-0e76e2a03463 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:08:20.791816 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:20.791816 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:20.791816 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:20.791816 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:20.791816 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:20.791816 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:20.791816 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:20.791816 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:20.791816 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:20.791816 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:20.791816 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:20.791816 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:08:20.791816 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:08:20.791816 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:08:20.791816 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:08:20.792819 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:08:20.792819 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:08:20.792819 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:08:20.792819 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:08:20.792819 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:20.792819 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:20.792819 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:20.792819 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:20.792819 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:20.792819 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:20.792819 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:20.792819 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:20.792819 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:20.792819 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:20.792819 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:20.792819 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:20.793711 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:20.793711 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:20.793711 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:20.793711 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:20.793711 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:08:20.793711 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:08:20.793711 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:08:20.793711 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:08:20.793711 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:08:20.793711 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:08:20.793711 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:08:20.793711 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:08:20.794339 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-8647b448-2c81-4636-bb94-2943ad93573d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:08:20.794339 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:20.794339 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:20.794339 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:20.794339 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:20.794339 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:20.794339 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:20.794339 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:20.794339 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:20.794339 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:20.794339 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:20.794339 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:20.794339 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:08:20.794339 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:08:20.794339 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:08:20.794339 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:08:20.796006 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:20.796006 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:20.796006 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:20.796006 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:20.796006 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:20.796006 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:20.796006 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:20.796006 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:20.796006 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:20.796006 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:20.796006 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:20.796006 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:20.796006 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:20.796006 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:20.796006 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:20.796006 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:20.796006 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:08:20.797009 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:08:20.797009 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:08:20.797009 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:08:20.797009 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 323/643] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 24 17:08:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:20.797009 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8647b448-2c81-4636-bb94-2943ad93573d None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:20.797723 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 321/644] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Aug 24 17:08:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:20.798737 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63eb2db7-33fc-4cbe-b1bb-0e76e2a03463 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:20.908225 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8647b448-2c81-4636-bb94-2943ad93573d None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:20.910951 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63eb2db7-33fc-4cbe-b1bb-0e76e2a03463 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:20.942208 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f47b9b7-f6ea-472d-9e24-90ce24a861b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:20.942818 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f47b9b7-f6ea-472d-9e24-90ce24a861b4 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:20.943143 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f47b9b7-f6ea-472d-9e24-90ce24a861b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:20.943417 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f47b9b7-f6ea-472d-9e24-90ce24a861b4 None admin] PATH_INFO: `/v3/domains` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:20.944059 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f47b9b7-f6ea-472d-9e24-90ce24a861b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:20.944362 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f47b9b7-f6ea-472d-9e24-90ce24a861b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:20.945156 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d720dfb-3e41-4199-8b62-b12276232aae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:20.945731 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f47b9b7-f6ea-472d-9e24-90ce24a861b4 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:20.945826 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d720dfb-3e41-4199-8b62-b12276232aae None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:20.946032 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d720dfb-3e41-4199-8b62-b12276232aae None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:20.946301 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d720dfb-3e41-4199-8b62-b12276232aae None admin] PATH_INFO: `/v3/projects/service` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:20.953582 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d720dfb-3e41-4199-8b62-b12276232aae None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:20.955020 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d720dfb-3e41-4199-8b62-b12276232aae None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:20.958946 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 324/645] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 24 17:08:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:20.963419 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f47b9b7-f6ea-472d-9e24-90ce24a861b4 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:20.966194 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-7d720dfb-3e41-4199-8b62-b12276232aae None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:08:20.966194 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:20.966194 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:20.966194 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:20.966194 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:20.966194 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:20.966194 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:20.966194 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:20.966194 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:20.966194 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:20.966194 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:20.966194 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:20.966194 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:08:20.966194 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:08:20.966194 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:08:20.967295 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:08:20.967295 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:20.967295 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:20.967295 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:20.967295 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:20.967295 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:20.967295 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:20.967295 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:20.967295 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:20.967295 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:20.967295 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:20.967295 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:20.967295 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:20.967295 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:20.967295 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:20.967295 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:20.967295 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:20.968190 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:08:20.968190 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:08:20.968190 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:08:20.968190 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:08:20.968190 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:08:20.968190 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:08:20.968190 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:08:20.968190 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:08:20.972110 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 322/646] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 24 17:08:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:20.972718 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d720dfb-3e41-4199-8b62-b12276232aae None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:20.987167 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:20.990477 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:21.063750 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f47b9b7-f6ea-472d-9e24-90ce24a861b4 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:21.081951 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d720dfb-3e41-4199-8b62-b12276232aae None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:21.099800 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550c18b2-1f92-4d9a-addb-b43e0ccf6e65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:21.100410 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550c18b2-1f92-4d9a-addb-b43e0ccf6e65 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:21.100689 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550c18b2-1f92-4d9a-addb-b43e0ccf6e65 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:21.100962 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550c18b2-1f92-4d9a-addb-b43e0ccf6e65 None admin] PATH_INFO: `/v3/domains` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:21.101599 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550c18b2-1f92-4d9a-addb-b43e0ccf6e65 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:21.101905 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550c18b2-1f92-4d9a-addb-b43e0ccf6e65 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:21.103295 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550c18b2-1f92-4d9a-addb-b43e0ccf6e65 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:21.114448 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 325/647] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 24 17:08:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:21.115323 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550c18b2-1f92-4d9a-addb-b43e0ccf6e65 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:21.126014 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0458b42-da52-4ea7-83fa-383d1c7ca477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:21.127341 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0458b42-da52-4ea7-83fa-383d1c7ca477 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:21.127763 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0458b42-da52-4ea7-83fa-383d1c7ca477 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:21.128584 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0458b42-da52-4ea7-83fa-383d1c7ca477 None admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:21.129682 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0458b42-da52-4ea7-83fa-383d1c7ca477 None admin] RBAC: Adding query filter params (name=service) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:21.130148 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0458b42-da52-4ea7-83fa-383d1c7ca477 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:21.132829 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0458b42-da52-4ea7-83fa-383d1c7ca477 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:21.145328 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 323/648] 10.222.0.18 () {56 vars in 1103 bytes} [Wed Aug 24 17:08:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:21.146257 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0458b42-da52-4ea7-83fa-383d1c7ca477 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:21.214592 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:21.223051 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:21.226292 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550c18b2-1f92-4d9a-addb-b43e0ccf6e65 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:21.255191 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0458b42-da52-4ea7-83fa-383d1c7ca477 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:21.263564 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6a412e-c0ea-4afb-83e1-285173a03986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:21.264205 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6a412e-c0ea-4afb-83e1-285173a03986 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:21.264727 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6a412e-c0ea-4afb-83e1-285173a03986 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:21.264727 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6a412e-c0ea-4afb-83e1-285173a03986 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:21.273361 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6a412e-c0ea-4afb-83e1-285173a03986 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:21.274823 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6a412e-c0ea-4afb-83e1-285173a03986 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:21.286868 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-0f6a412e-c0ea-4afb-83e1-285173a03986 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 24 17:08:21.286868 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:21.286868 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:21.286868 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:21.286868 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:21.286868 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:21.286868 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:21.286868 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:21.286868 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:21.286868 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:21.286868 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:21.286868 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:21.286868 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:08:21.286868 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:08:21.286868 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:08:21.286868 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:08:21.287927 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:21.287927 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:21.287927 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:08:21.287927 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:08:21.287927 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:08:21.287927 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:08:21.287927 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:21.287927 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:21.287927 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:21.287927 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:21.287927 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:21.287927 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:21.287927 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:21.287927 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:21.287927 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:21.287927 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:21.287927 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:21.287927 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:21.288880 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:21.288880 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:21.288880 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:08:21.288880 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:08:21.288880 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:08:21.288880 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:08:21.288880 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:08:21.288880 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:08:21.288880 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 24 17:08:21.288880 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:08:21.288880 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 326/649] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 24 17:08:21 2022] GET /identity/v3/users/glance => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:21.288880 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6a412e-c0ea-4afb-83e1-285173a03986 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:21.297795 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0f23fa-4140-49a1-ab1f-c267c1f9d4a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:21.298406 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0f23fa-4140-49a1-ab1f-c267c1f9d4a5 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:21.298689 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0f23fa-4140-49a1-ab1f-c267c1f9d4a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:21.298993 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0f23fa-4140-49a1-ab1f-c267c1f9d4a5 None admin] PATH_INFO: `/v3/projects` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:21.299688 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0f23fa-4140-49a1-ab1f-c267c1f9d4a5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:21.299992 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0f23fa-4140-49a1-ab1f-c267c1f9d4a5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:21.301431 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0f23fa-4140-49a1-ab1f-c267c1f9d4a5 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:21.314959 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 324/650] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Aug 24 17:08:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:21.315770 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0f23fa-4140-49a1-ab1f-c267c1f9d4a5 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:21.404034 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6a412e-c0ea-4afb-83e1-285173a03986 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:21.423888 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0f23fa-4140-49a1-ab1f-c267c1f9d4a5 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:21.438751 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48935ad0-340e-408d-855f-ca77faefc792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:21.439409 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48935ad0-340e-408d-855f-ca77faefc792 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:21.439673 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48935ad0-340e-408d-855f-ca77faefc792 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:21.439936 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48935ad0-340e-408d-855f-ca77faefc792 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:21.447903 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48935ad0-340e-408d-855f-ca77faefc792 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:21.449510 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48935ad0-340e-408d-855f-ca77faefc792 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:21.455063 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:21.460963 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-48935ad0-340e-408d-855f-ca77faefc792 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 17:08:21.460963 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:21.460963 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:21.460963 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:21.460963 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:21.460963 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:21.460963 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:21.460963 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:21.460963 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:21.460963 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:21.460963 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:21.460963 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:21.460963 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:08:21.460963 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:08:21.460963 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:08:21.460963 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:08:21.461873 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:21.461873 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:21.461873 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:08:21.461873 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:08:21.461873 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:08:21.461873 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:08:21.461873 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:21.461873 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:21.461873 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:21.461873 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:21.461873 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:21.461873 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:21.461873 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:21.461873 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:21.461873 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:21.461873 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:21.461873 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:21.461873 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:21.462812 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:21.462812 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:21.462812 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:08:21.462812 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:08:21.462812 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:08:21.462812 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:08:21.462812 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:08:21.462812 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:08:21.462812 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 17:08:21.462812 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:08:21.468247 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934f36d2-0add-46b5-8421-1430f5e01fc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:21.468247 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934f36d2-0add-46b5-8421-1430f5e01fc2 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:21.468247 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934f36d2-0add-46b5-8421-1430f5e01fc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:21.468247 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934f36d2-0add-46b5-8421-1430f5e01fc2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:21.468565 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 327/651] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Aug 24 17:08:21 2022] GET /identity/v3/users/neutron => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:21.468565 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48935ad0-340e-408d-855f-ca77faefc792 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:21.478035 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934f36d2-0add-46b5-8421-1430f5e01fc2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:21.478493 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:21.479853 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934f36d2-0add-46b5-8421-1430f5e01fc2 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:21.496304 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-934f36d2-0add-46b5-8421-1430f5e01fc2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:08:21.496304 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:21.496304 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:21.496304 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:21.496304 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:21.496304 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:21.496304 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:21.496304 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:21.496304 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:21.496304 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:21.496304 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:21.496304 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:21.496304 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:08:21.496304 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:08:21.496304 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:08:21.496304 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:08:21.498026 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:21.498026 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:21.498026 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:21.498026 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:21.498026 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:21.498026 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:21.498026 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:21.498026 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:21.498026 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:21.498026 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:21.498026 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:21.498026 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:21.498026 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:21.498026 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:21.498026 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:21.498026 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:21.498026 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:08:21.499161 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:08:21.499161 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:08:21.499161 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:08:21.503829 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 325/652] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 24 17:08:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:21.504040 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934f36d2-0add-46b5-8421-1430f5e01fc2 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:21.591428 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48935ad0-340e-408d-855f-ca77faefc792 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:21.617109 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934f36d2-0add-46b5-8421-1430f5e01fc2 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:21.630127 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af20489f-7181-49ad-b4eb-ccf55cd209bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:21.631053 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af20489f-7181-49ad-b4eb-ccf55cd209bd None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:21.631560 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af20489f-7181-49ad-b4eb-ccf55cd209bd None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:21.632025 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af20489f-7181-49ad-b4eb-ccf55cd209bd None admin] PATH_INFO: `/v3/users` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:21.632988 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af20489f-7181-49ad-b4eb-ccf55cd209bd None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:21.633400 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af20489f-7181-49ad-b4eb-ccf55cd209bd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:21.634896 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af20489f-7181-49ad-b4eb-ccf55cd209bd None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:21.657519 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8473405-b981-42fc-a9ec-72f184d08ca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:21.657722 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 328/653] 10.222.0.18 () {56 vars in 1137 bytes} [Wed Aug 24 17:08:21 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:21.657817 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8473405-b981-42fc-a9ec-72f184d08ca2 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:21.657817 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8473405-b981-42fc-a9ec-72f184d08ca2 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:21.657971 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af20489f-7181-49ad-b4eb-ccf55cd209bd None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:21.658077 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8473405-b981-42fc-a9ec-72f184d08ca2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:21.658077 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8473405-b981-42fc-a9ec-72f184d08ca2 None admin] RBAC: Adding query filter params (role.id=e7db2a847f56438bac13d0ff21c96b73, scope.project.id=46ab158bc3644c888557781b0630b0b8, user.id=557aca9255cb4bbb8af52e796c5c5177) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:21.658077 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8473405-b981-42fc-a9ec-72f184d08ca2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:21.658077 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8473405-b981-42fc-a9ec-72f184d08ca2 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:21.666933 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 326/654] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 24 17:08:21 2022] GET /identity/v3/role_assignments?user.id=557aca9255cb4bbb8af52e796c5c5177&scope.project.id=46ab158bc3644c888557781b0630b0b8&role.id=e7db2a847f56438bac13d0ff21c96b73 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:21.666933 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8473405-b981-42fc-a9ec-72f184d08ca2 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:21.734524 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:21.763242 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:21.774495 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af20489f-7181-49ad-b4eb-ccf55cd209bd None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:21.817905 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad6d84f-4c32-4f61-9233-6413c97b3a94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:21.818525 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad6d84f-4c32-4f61-9233-6413c97b3a94 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:21.819930 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad6d84f-4c32-4f61-9233-6413c97b3a94 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:21.819930 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad6d84f-4c32-4f61-9233-6413c97b3a94 None admin] PATH_INFO: `/v3/users` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:21.819930 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad6d84f-4c32-4f61-9233-6413c97b3a94 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:21.820153 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad6d84f-4c32-4f61-9233-6413c97b3a94 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:21.820699 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8473405-b981-42fc-a9ec-72f184d08ca2 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:21.821585 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad6d84f-4c32-4f61-9233-6413c97b3a94 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:21.842927 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 329/655] 10.222.0.18 () {56 vars in 1139 bytes} [Wed Aug 24 17:08:21 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:21.843802 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad6d84f-4c32-4f61-9233-6413c97b3a94 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:21.864772 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f336e5d7-f1b3-4f1f-8127-6cd9adf03e0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:21.865381 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f336e5d7-f1b3-4f1f-8127-6cd9adf03e0c None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:21.866466 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f336e5d7-f1b3-4f1f-8127-6cd9adf03e0c None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:21.866749 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f336e5d7-f1b3-4f1f-8127-6cd9adf03e0c None admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:21.867423 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f336e5d7-f1b3-4f1f-8127-6cd9adf03e0c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:21.867726 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f336e5d7-f1b3-4f1f-8127-6cd9adf03e0c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:21.869092 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f336e5d7-f1b3-4f1f-8127-6cd9adf03e0c None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:21.882818 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 327/656] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 24 17:08:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:21.884473 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f336e5d7-f1b3-4f1f-8127-6cd9adf03e0c None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:21.969277 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad6d84f-4c32-4f61-9233-6413c97b3a94 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:22.000384 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:22.009829 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f336e5d7-f1b3-4f1f-8127-6cd9adf03e0c None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:22.020190 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c071a13-0ae4-4387-b923-18b39293cb1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:22.020190 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c071a13-0ae4-4387-b923-18b39293cb1a None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:22.020190 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c071a13-0ae4-4387-b923-18b39293cb1a None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:22.020190 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c071a13-0ae4-4387-b923-18b39293cb1a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:22.023916 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:22.060560 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c9b211-78f2-49c1-a212-4867956f78a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:22.061976 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c9b211-78f2-49c1-a212-4867956f78a6 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:22.062559 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c9b211-78f2-49c1-a212-4867956f78a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:22.063485 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c9b211-78f2-49c1-a212-4867956f78a6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:22.094855 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c071a13-0ae4-4387-b923-18b39293cb1a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:22.096337 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c071a13-0ae4-4387-b923-18b39293cb1a None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:22.150878 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c9b211-78f2-49c1-a212-4867956f78a6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:22.152362 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c9b211-78f2-49c1-a212-4867956f78a6 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:22.203928 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 330/657] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 24 17:08:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 17:08:22.205100 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c071a13-0ae4-4387-b923-18b39293cb1a None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:22.232674 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:22.259723 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 328/658] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 24 17:08:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 17:08:22.261275 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a62a18-0775-407f-b844-a958d150d164 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:22.261644 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a62a18-0775-407f-b844-a958d150d164 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:22.261992 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a62a18-0775-407f-b844-a958d150d164 None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:22.266938 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 329/659] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:08:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:08:22.266938 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a62a18-0775-407f-b844-a958d150d164 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:22.276897 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:22.320843 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c071a13-0ae4-4387-b923-18b39293cb1a None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:22.356590 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80009722-31de-41e8-bd4a-2e68ddb62032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:22.357215 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80009722-31de-41e8-bd4a-2e68ddb62032 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:22.357496 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80009722-31de-41e8-bd4a-2e68ddb62032 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:22.357770 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80009722-31de-41e8-bd4a-2e68ddb62032 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:22.365047 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80009722-31de-41e8-bd4a-2e68ddb62032 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:22.366483 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80009722-31de-41e8-bd4a-2e68ddb62032 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:22.378594 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-80009722-31de-41e8-bd4a-2e68ddb62032 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 17:08:22.378594 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:22.378594 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:22.378594 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:22.378594 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:22.378594 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:22.378594 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:22.378594 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:22.378594 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:22.378594 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:22.378594 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:22.378594 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:22.378594 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:08:22.378594 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:08:22.378594 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:08:22.378594 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:08:22.383789 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:22.383789 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:22.383789 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:08:22.383789 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:08:22.383789 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:08:22.383789 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:08:22.383789 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:22.383789 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:22.383789 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:22.383789 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:22.383789 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:22.383789 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:22.383789 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:22.383789 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:22.383789 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:22.383789 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:22.383789 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:22.383789 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:22.384728 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:22.384728 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:22.384728 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:08:22.384728 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:08:22.384728 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:08:22.384728 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:08:22.384728 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:08:22.384728 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:08:22.384728 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 17:08:22.384728 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:08:22.384728 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 331/660] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 24 17:08:22 2022] GET /identity/v3/users/cinder => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:22.384728 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80009722-31de-41e8-bd4a-2e68ddb62032 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:22.386708 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a62a18-0775-407f-b844-a958d150d164 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:22.424207 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e985c8-efad-4faa-84c0-a31813b9ceb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:22.425092 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e985c8-efad-4faa-84c0-a31813b9ceb2 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:22.425515 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e985c8-efad-4faa-84c0-a31813b9ceb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:22.425948 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e985c8-efad-4faa-84c0-a31813b9ceb2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:22.437322 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e985c8-efad-4faa-84c0-a31813b9ceb2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:22.438778 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e985c8-efad-4faa-84c0-a31813b9ceb2 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:22.453960 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-d5e985c8-efad-4faa-84c0-a31813b9ceb2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:08:22.453960 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:22.453960 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:22.453960 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:22.453960 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:22.453960 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:22.453960 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:22.453960 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:22.453960 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:22.453960 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:22.453960 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:22.453960 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:22.453960 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:08:22.453960 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:08:22.453960 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:08:22.453960 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:08:22.455135 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:22.455135 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:22.455135 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:22.455135 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:22.455135 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:22.455135 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:22.455135 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:22.455135 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:22.455135 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:22.455135 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:22.455135 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:22.455135 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:22.455135 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:22.455135 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:22.455135 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:22.455135 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:22.455135 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:08:22.456227 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:08:22.456227 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:08:22.456227 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:08:22.459582 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 330/661] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 24 17:08:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:22.460878 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751c945f-6a60-4762-bff9-52f7869d8733 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:22.461174 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751c945f-6a60-4762-bff9-52f7869d8733 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:22.461454 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751c945f-6a60-4762-bff9-52f7869d8733 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:22.470662 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:22.479017 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-751c945f-6a60-4762-bff9-52f7869d8733 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:08:22.480216 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-751c945f-6a60-4762-bff9-52f7869d8733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:08:22.488254 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80009722-31de-41e8-bd4a-2e68ddb62032 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:22.523600 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fd161c-5742-44c4-9965-495ba124420a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:22.524158 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fd161c-5742-44c4-9965-495ba124420a None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:22.524440 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fd161c-5742-44c4-9965-495ba124420a None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:22.524713 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fd161c-5742-44c4-9965-495ba124420a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:22.534974 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fd161c-5742-44c4-9965-495ba124420a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:22.536411 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fd161c-5742-44c4-9965-495ba124420a None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:22.539509 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:22.553415 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-a8fd161c-5742-44c4-9965-495ba124420a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:08:22.553415 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:22.553415 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:22.553415 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:22.553415 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:22.553415 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:22.553415 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:22.553415 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:22.553415 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:22.553415 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:22.553415 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:22.553415 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:22.553415 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:08:22.553415 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:08:22.553415 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:08:22.553415 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:08:22.554868 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:22.554868 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:22.554868 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:22.554868 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:22.554868 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:22.554868 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:22.554868 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:22.554868 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:22.554868 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:22.554868 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:22.554868 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:22.554868 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:22.554868 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:22.554868 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:22.554868 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:22.554868 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:22.554868 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:08:22.556146 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:08:22.556146 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:08:22.556146 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:08:22.556146 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 332/662] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 24 17:08:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:22.556146 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fd161c-5742-44c4-9965-495ba124420a None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:22.564124 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 331/663] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:08:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:08:22.564824 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751c945f-6a60-4762-bff9-52f7869d8733 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:22.671796 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751c945f-6a60-4762-bff9-52f7869d8733 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:22.683974 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fd161c-5742-44c4-9965-495ba124420a None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:22.707740 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bad43b-4181-4b3e-8c28-d62e43739acb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:22.708462 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bad43b-4181-4b3e-8c28-d62e43739acb None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:22.708673 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bad43b-4181-4b3e-8c28-d62e43739acb None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:22.708953 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bad43b-4181-4b3e-8c28-d62e43739acb None admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:22.709601 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bad43b-4181-4b3e-8c28-d62e43739acb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:22.709904 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bad43b-4181-4b3e-8c28-d62e43739acb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:22.711417 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bad43b-4181-4b3e-8c28-d62e43739acb None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:22.723000 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac84b6ce-0f07-4626-ba69-f9364fdad688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:22.723931 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac84b6ce-0f07-4626-ba69-f9364fdad688 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:22.724426 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac84b6ce-0f07-4626-ba69-f9364fdad688 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:22.724915 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac84b6ce-0f07-4626-ba69-f9364fdad688 None admin] PATH_INFO: `/v3/users` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:22.725835 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac84b6ce-0f07-4626-ba69-f9364fdad688 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:22.726348 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac84b6ce-0f07-4626-ba69-f9364fdad688 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:22.727975 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac84b6ce-0f07-4626-ba69-f9364fdad688 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:22.728086 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 332/664] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 24 17:08:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:22.728955 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bad43b-4181-4b3e-8c28-d62e43739acb None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:22.731691 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:22.754675 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 333/665] 10.222.0.18 () {56 vars in 1137 bytes} [Wed Aug 24 17:08:22 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:22.755849 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac84b6ce-0f07-4626-ba69-f9364fdad688 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:22.790782 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:22.846812 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bad43b-4181-4b3e-8c28-d62e43739acb None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:22.867350 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac84b6ce-0f07-4626-ba69-f9364fdad688 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:22.883162 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caed6f42-ca56-4d49-a438-dc94b186beff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:22.883770 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caed6f42-ca56-4d49-a438-dc94b186beff None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:22.884053 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caed6f42-ca56-4d49-a438-dc94b186beff None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:22.884338 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caed6f42-ca56-4d49-a438-dc94b186beff None admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:22.884969 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caed6f42-ca56-4d49-a438-dc94b186beff None admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:22.885275 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caed6f42-ca56-4d49-a438-dc94b186beff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:22.886649 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caed6f42-ca56-4d49-a438-dc94b186beff None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:22.898679 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 333/666] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 24 17:08:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:22.899593 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caed6f42-ca56-4d49-a438-dc94b186beff None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:22.915763 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0788caa-0410-437c-90b0-ab263fb2bb00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:22.917164 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0788caa-0410-437c-90b0-ab263fb2bb00 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:22.917911 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0788caa-0410-437c-90b0-ab263fb2bb00 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:22.918200 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0788caa-0410-437c-90b0-ab263fb2bb00 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:22.926590 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0788caa-0410-437c-90b0-ab263fb2bb00 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:22.928054 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0788caa-0410-437c-90b0-ab263fb2bb00 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:22.939507 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-d0788caa-0410-437c-90b0-ab263fb2bb00 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:08:22.939507 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:22.939507 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:22.939507 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:22.939507 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:22.939507 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:22.939507 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:22.939507 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:22.939507 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:22.939507 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:22.939507 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:22.939507 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:22.939507 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:08:22.939507 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:08:22.939507 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:08:22.940388 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:08:22.940388 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:22.940388 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:22.940388 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:22.940388 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:22.940388 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:22.940388 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:22.940388 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:22.940388 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:22.940388 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:22.940388 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:22.940388 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:22.940388 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:22.940388 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:22.940388 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:22.940388 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:22.940388 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:22.941314 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:08:22.941314 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:08:22.941314 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:08:22.941314 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:08:22.941314 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:08:22.941314 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:08:22.941314 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:08:22.941314 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:08:22.941314 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 334/667] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 24 17:08:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:22.941314 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0788caa-0410-437c-90b0-ab263fb2bb00 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:22.999566 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:23.014335 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caed6f42-ca56-4d49-a438-dc94b186beff None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:23.025454 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:23.045353 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0788caa-0410-437c-90b0-ab263fb2bb00 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:23.049859 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53933f4-063e-4f76-858a-d68b65d2c0a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:23.050464 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53933f4-063e-4f76-858a-d68b65d2c0a4 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:23.050746 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53933f4-063e-4f76-858a-d68b65d2c0a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:23.051056 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53933f4-063e-4f76-858a-d68b65d2c0a4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:23.081045 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef24d5e0-0818-45fb-b317-e30d0add5250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:23.081661 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef24d5e0-0818-45fb-b317-e30d0add5250 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:23.081938 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef24d5e0-0818-45fb-b317-e30d0add5250 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:23.082211 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef24d5e0-0818-45fb-b317-e30d0add5250 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:23.093042 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef24d5e0-0818-45fb-b317-e30d0add5250 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:08:23.097743 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-ef24d5e0-0818-45fb-b317-e30d0add5250 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:08:23.097743 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:23.097743 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:23.097743 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:23.097743 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:23.097743 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:23.097743 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:23.097743 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:23.097743 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:23.097743 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:23.097743 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:23.097743 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:23.097743 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:08:23.097743 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:08:23.097743 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:08:23.097743 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:08:23.098666 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:08:23.098666 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:08:23.098666 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:08:23.098666 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:08:23.098666 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:23.098666 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:23.098666 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:23.098666 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:23.098666 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:23.098666 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:23.098666 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:23.098666 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:23.098666 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:23.098666 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:23.098666 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:23.098666 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:23.099581 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:23.099581 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:23.099581 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:23.099581 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:23.099581 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:08:23.099581 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:08:23.099581 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:08:23.099581 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:08:23.099581 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:08:23.099581 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:08:23.099581 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:08:23.099581 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:08:23.099581 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 335/668] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Aug 24 17:08:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:23.099581 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef24d5e0-0818-45fb-b317-e30d0add5250 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:23.128171 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53933f4-063e-4f76-858a-d68b65d2c0a4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:23.129857 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53933f4-063e-4f76-858a-d68b65d2c0a4 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:23.208559 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef24d5e0-0818-45fb-b317-e30d0add5250 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:23.223717 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:23.242041 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 334/669] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 24 17:08:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 17:08:23.242041 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53933f4-063e-4f76-858a-d68b65d2c0a4 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:23.249676 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c89624c-b3f2-49dd-ad82-890081eb81ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:23.250312 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c89624c-b3f2-49dd-ad82-890081eb81ba None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:23.250581 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c89624c-b3f2-49dd-ad82-890081eb81ba None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:23.251775 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c89624c-b3f2-49dd-ad82-890081eb81ba None admin] PATH_INFO: `/v3/projects/service` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:23.253408 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:23.259158 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c89624c-b3f2-49dd-ad82-890081eb81ba None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:23.264680 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c89624c-b3f2-49dd-ad82-890081eb81ba None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:23.272285 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-1c89624c-b3f2-49dd-ad82-890081eb81ba None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:08:23.272285 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:23.272285 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:23.272285 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:23.272285 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:23.272285 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:23.272285 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:23.272285 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:23.272285 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:23.272285 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:23.272285 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:23.272285 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:23.272285 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:08:23.272285 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:08:23.272285 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:08:23.273379 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:08:23.273379 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:23.273379 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:23.273379 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:23.273379 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:23.273379 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:23.273379 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:23.273379 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:23.273379 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:23.273379 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:23.273379 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:23.273379 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:23.273379 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:23.273379 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:23.273379 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:23.273379 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:23.273379 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:23.274292 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:08:23.274292 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:08:23.274292 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:08:23.274292 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:08:23.274292 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:08:23.274292 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:08:23.274292 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:08:23.274292 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:08:23.274292 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 336/670] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 24 17:08:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:23.274771 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c89624c-b3f2-49dd-ad82-890081eb81ba None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:23.350784 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53933f4-063e-4f76-858a-d68b65d2c0a4 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:23.379496 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c89624c-b3f2-49dd-ad82-890081eb81ba None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:23.389670 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b3a8a6-fb83-4a51-9c86-238f6d2479be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:23.390641 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b3a8a6-fb83-4a51-9c86-238f6d2479be None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:23.390950 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b3a8a6-fb83-4a51-9c86-238f6d2479be None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:23.391232 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b3a8a6-fb83-4a51-9c86-238f6d2479be None admin] PATH_INFO: `/v3/projects` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:23.391933 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b3a8a6-fb83-4a51-9c86-238f6d2479be None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:23.392247 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b3a8a6-fb83-4a51-9c86-238f6d2479be None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:23.393987 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b3a8a6-fb83-4a51-9c86-238f6d2479be None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:23.405953 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 335/671] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Aug 24 17:08:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:23.406765 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b3a8a6-fb83-4a51-9c86-238f6d2479be None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:23.422429 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093b8d0e-21d7-4a53-b63b-660b097782b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:23.423322 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093b8d0e-21d7-4a53-b63b-660b097782b6 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:23.423860 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093b8d0e-21d7-4a53-b63b-660b097782b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:23.424273 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093b8d0e-21d7-4a53-b63b-660b097782b6 None admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:23.425180 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093b8d0e-21d7-4a53-b63b-660b097782b6 None admin] RBAC: Adding query filter params (name=service) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:23.425621 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093b8d0e-21d7-4a53-b63b-660b097782b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:23.427262 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093b8d0e-21d7-4a53-b63b-660b097782b6 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:23.436591 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 337/672] 10.222.0.18 () {56 vars in 1103 bytes} [Wed Aug 24 17:08:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:23.437598 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093b8d0e-21d7-4a53-b63b-660b097782b6 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:23.476736 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:23.482950 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:23.515049 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b3a8a6-fb83-4a51-9c86-238f6d2479be None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:23.563924 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093b8d0e-21d7-4a53-b63b-660b097782b6 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:23.568109 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b5b915-aff3-46bb-b53d-bc1eb32015aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:23.568751 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b5b915-aff3-46bb-b53d-bc1eb32015aa None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:23.569030 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b5b915-aff3-46bb-b53d-bc1eb32015aa None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:23.569302 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b5b915-aff3-46bb-b53d-bc1eb32015aa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:23.581053 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b5b915-aff3-46bb-b53d-bc1eb32015aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:23.582617 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b5b915-aff3-46bb-b53d-bc1eb32015aa None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:23.600067 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-d6b5b915-aff3-46bb-b53d-bc1eb32015aa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:08:23.600067 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:23.600067 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:23.600067 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:23.600067 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:23.600067 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:23.600067 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:23.600067 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:23.600067 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:23.600067 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:23.600067 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:23.600067 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:23.600067 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:08:23.600067 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:08:23.600067 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:08:23.600067 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:08:23.601124 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:23.601124 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:23.601124 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:23.601124 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:23.601124 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:23.601124 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:23.601124 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:23.601124 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:23.601124 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:23.601124 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:23.601124 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:23.601124 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:23.601124 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:23.601124 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:23.601124 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:23.601124 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:23.601124 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:08:23.602017 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:08:23.602017 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:08:23.602017 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:08:23.602189 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cab219-02d6-4df7-bbe0-04bc062e41b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:23.602189 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cab219-02d6-4df7-bbe0-04bc062e41b0 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:23.602189 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cab219-02d6-4df7-bbe0-04bc062e41b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:23.602189 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cab219-02d6-4df7-bbe0-04bc062e41b0 None admin] PATH_INFO: `/v3/projects` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:23.602995 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cab219-02d6-4df7-bbe0-04bc062e41b0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:23.602995 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cab219-02d6-4df7-bbe0-04bc062e41b0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:23.604352 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cab219-02d6-4df7-bbe0-04bc062e41b0 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:23.605734 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 336/673] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 24 17:08:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:23.607439 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b5b915-aff3-46bb-b53d-bc1eb32015aa None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:23.625026 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 338/674] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Aug 24 17:08:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:23.625026 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cab219-02d6-4df7-bbe0-04bc062e41b0 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:23.720146 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b5b915-aff3-46bb-b53d-bc1eb32015aa None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:23.721066 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:23.727971 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cab219-02d6-4df7-bbe0-04bc062e41b0 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:23.743159 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:23.755499 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ffd654-bc34-424d-90b5-f8eae4ae167a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:23.756101 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ffd654-bc34-424d-90b5-f8eae4ae167a None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:23.756378 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ffd654-bc34-424d-90b5-f8eae4ae167a None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:23.756790 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ffd654-bc34-424d-90b5-f8eae4ae167a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:23.757543 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ffd654-bc34-424d-90b5-f8eae4ae167a None admin] RBAC: Adding query filter params (role.id=e7db2a847f56438bac13d0ff21c96b73, scope.project.id=46ab158bc3644c888557781b0630b0b8, user.id=3553e522363046f6828fd61b2a0642ed) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:23.757859 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ffd654-bc34-424d-90b5-f8eae4ae167a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:23.759289 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ffd654-bc34-424d-90b5-f8eae4ae167a None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:23.763383 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589f3fb5-63f8-4c13-9502-1a8ace624ab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:23.764234 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589f3fb5-63f8-4c13-9502-1a8ace624ab4 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:23.764746 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589f3fb5-63f8-4c13-9502-1a8ace624ab4 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:23.765302 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 337/675] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 24 17:08:23 2022] GET /identity/v3/role_assignments?user.id=3553e522363046f6828fd61b2a0642ed&scope.project.id=46ab158bc3644c888557781b0630b0b8&role.id=e7db2a847f56438bac13d0ff21c96b73 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:23.765427 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589f3fb5-63f8-4c13-9502-1a8ace624ab4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:23.765881 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ffd654-bc34-424d-90b5-f8eae4ae167a None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:23.785022 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589f3fb5-63f8-4c13-9502-1a8ace624ab4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:23.786502 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589f3fb5-63f8-4c13-9502-1a8ace624ab4 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:23.803730 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-589f3fb5-63f8-4c13-9502-1a8ace624ab4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:08:23.803730 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:23.803730 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:23.803730 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:23.803730 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:23.803730 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:23.803730 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:23.803730 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:23.803730 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:23.803730 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:23.803730 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:23.803730 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:23.803730 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:08:23.803730 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:08:23.803730 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:08:23.803730 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:08:23.804782 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:23.804782 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:23.804782 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:23.804782 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:23.804782 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:23.804782 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:23.804782 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:23.804782 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:23.804782 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:23.804782 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:23.804782 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:23.804782 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:23.804782 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:23.804782 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:23.804782 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:23.804782 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:23.804782 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:08:23.805673 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:08:23.805673 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:08:23.805673 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:08:23.805673 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 339/676] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 24 17:08:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:23.805673 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589f3fb5-63f8-4c13-9502-1a8ace624ab4 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:23.887137 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ffd654-bc34-424d-90b5-f8eae4ae167a None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:23.920314 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589f3fb5-63f8-4c13-9502-1a8ace624ab4 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:23.926975 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419ca6c7-d4b9-485a-95f9-fe459fc7460d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:23.927582 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419ca6c7-d4b9-485a-95f9-fe459fc7460d None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:23.927866 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419ca6c7-d4b9-485a-95f9-fe459fc7460d None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:23.928147 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419ca6c7-d4b9-485a-95f9-fe459fc7460d None admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:23.928795 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419ca6c7-d4b9-485a-95f9-fe459fc7460d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:23.929103 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419ca6c7-d4b9-485a-95f9-fe459fc7460d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:23.930537 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419ca6c7-d4b9-485a-95f9-fe459fc7460d None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:23.942944 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 338/677] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 24 17:08:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:23.944680 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d315f3dd-198a-4bd6-af5c-a3e02cbfea8e None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:23.944796 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d315f3dd-198a-4bd6-af5c-a3e02cbfea8e None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:23.945038 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d315f3dd-198a-4bd6-af5c-a3e02cbfea8e None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:23.946434 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 339/678] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:08:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:08:23.946995 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d315f3dd-198a-4bd6-af5c-a3e02cbfea8e None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:23.958263 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fce6035-b38a-4054-b9b4-209d2a17b86c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:23.959567 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fce6035-b38a-4054-b9b4-209d2a17b86c None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:23.959567 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fce6035-b38a-4054-b9b4-209d2a17b86c None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:23.959567 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fce6035-b38a-4054-b9b4-209d2a17b86c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:23.960206 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fce6035-b38a-4054-b9b4-209d2a17b86c None admin] RBAC: Adding query filter params (role.id=e7db2a847f56438bac13d0ff21c96b73, scope.project.id=46ab158bc3644c888557781b0630b0b8, user.id=0e9ed4d47aef4140bfb2d3bb798cc559) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:23.960514 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fce6035-b38a-4054-b9b4-209d2a17b86c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:23.961940 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fce6035-b38a-4054-b9b4-209d2a17b86c None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:23.968313 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 340/679] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 24 17:08:23 2022] GET /identity/v3/role_assignments?user.id=0e9ed4d47aef4140bfb2d3bb798cc559&scope.project.id=46ab158bc3644c888557781b0630b0b8&role.id=e7db2a847f56438bac13d0ff21c96b73 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:23.969100 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fce6035-b38a-4054-b9b4-209d2a17b86c None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:23.998782 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:24.002089 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:24.088315 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d315f3dd-198a-4bd6-af5c-a3e02cbfea8e None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:24.108851 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fce6035-b38a-4054-b9b4-209d2a17b86c None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:24.129033 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e54c30-3874-4ccc-ae5c-9e022a1687e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:24.129639 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e54c30-3874-4ccc-ae5c-9e022a1687e2 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:24.129913 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e54c30-3874-4ccc-ae5c-9e022a1687e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:24.130179 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e54c30-3874-4ccc-ae5c-9e022a1687e2 None admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:24.130816 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e54c30-3874-4ccc-ae5c-9e022a1687e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:24.131779 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e54c30-3874-4ccc-ae5c-9e022a1687e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:24.133136 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e54c30-3874-4ccc-ae5c-9e022a1687e2 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:24.150249 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 340/680] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 24 17:08:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:24.151628 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c77f2c-3876-47b8-b5ba-d54e08eac8b8 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:24.151917 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c77f2c-3876-47b8-b5ba-d54e08eac8b8 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:24.152190 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c77f2c-3876-47b8-b5ba-d54e08eac8b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:24.158864 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a461aa-1f7f-4684-8126-e4334a4eca9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:24.159925 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a461aa-1f7f-4684-8126-e4334a4eca9e None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:24.160206 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a461aa-1f7f-4684-8126-e4334a4eca9e None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:24.160481 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a461aa-1f7f-4684-8126-e4334a4eca9e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:24.169216 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a461aa-1f7f-4684-8126-e4334a4eca9e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:24.170682 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a461aa-1f7f-4684-8126-e4334a4eca9e None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:24.172122 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-19c77f2c-3876-47b8-b5ba-d54e08eac8b8 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:08:24.173621 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-19c77f2c-3876-47b8-b5ba-d54e08eac8b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:08:24.185849 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-60a461aa-1f7f-4684-8126-e4334a4eca9e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:08:24.185849 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:24.185849 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:24.185849 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:24.185849 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:24.185849 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:24.185849 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:24.185849 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:24.185849 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:24.185849 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:24.185849 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:24.185849 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:24.185849 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:08:24.185849 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:08:24.185849 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:08:24.186816 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:08:24.186816 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:24.186816 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:24.186816 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:24.186816 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:24.186816 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:24.186816 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:24.186816 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:24.186816 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:24.186816 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:24.186816 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:24.186816 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:24.186816 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:24.186816 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:24.186816 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:24.186816 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:24.186816 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:24.187747 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:08:24.187747 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:08:24.187747 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:08:24.187747 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:08:24.187747 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:08:24.187747 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:08:24.187747 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:08:24.187747 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:08:24.190095 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 341/681] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 24 17:08:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:24.191254 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a461aa-1f7f-4684-8126-e4334a4eca9e None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:24.255559 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 341/682] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:08:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:08:24.256721 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c77f2c-3876-47b8-b5ba-d54e08eac8b8 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:24.271356 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:24.302138 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a461aa-1f7f-4684-8126-e4334a4eca9e None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:24.314099 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:24.338597 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383053e9-0bce-4694-a00d-7a4cd5d48e1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:24.343889 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383053e9-0bce-4694-a00d-7a4cd5d48e1e None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:24.344199 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383053e9-0bce-4694-a00d-7a4cd5d48e1e None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:24.344459 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383053e9-0bce-4694-a00d-7a4cd5d48e1e None admin] PATH_INFO: `/v3/users/swift` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:24.364010 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c77f2c-3876-47b8-b5ba-d54e08eac8b8 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:24.365359 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383053e9-0bce-4694-a00d-7a4cd5d48e1e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:24.368548 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383053e9-0bce-4694-a00d-7a4cd5d48e1e None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:24.391133 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-383053e9-0bce-4694-a00d-7a4cd5d48e1e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 17:08:24.391133 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:24.391133 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:24.391133 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:24.391133 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:24.391133 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:24.391133 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:24.391133 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:24.391133 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:24.391133 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:24.391133 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:24.391133 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:24.391133 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:08:24.391133 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:08:24.391133 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:08:24.391133 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:08:24.392268 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:24.392268 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:24.392268 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:08:24.392268 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:08:24.392268 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:08:24.392268 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:08:24.392268 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:24.392268 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:24.392268 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:24.392268 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:24.392268 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:24.392268 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:24.392268 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:24.392268 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:24.392268 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:24.392268 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:24.392268 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:24.392268 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:24.393237 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:24.393237 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:24.393237 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:08:24.393237 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:08:24.393237 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:08:24.393237 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:08:24.393237 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:08:24.393237 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:08:24.393237 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 17:08:24.393237 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:08:24.399260 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 342/683] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Aug 24 17:08:24 2022] GET /identity/v3/users/swift => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:24.407026 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383053e9-0bce-4694-a00d-7a4cd5d48e1e None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:24.459648 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3ade3a-e711-4020-91d4-968034b035c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:24.460277 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3ade3a-e711-4020-91d4-968034b035c4 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:24.460555 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3ade3a-e711-4020-91d4-968034b035c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:24.460823 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3ade3a-e711-4020-91d4-968034b035c4 None admin] PATH_INFO: `/v3/projects` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:24.461606 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3ade3a-e711-4020-91d4-968034b035c4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:24.461818 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3ade3a-e711-4020-91d4-968034b035c4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:24.463274 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3ade3a-e711-4020-91d4-968034b035c4 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:24.489169 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 342/684] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Aug 24 17:08:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:24.495219 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3ade3a-e711-4020-91d4-968034b035c4 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:24.528776 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383053e9-0bce-4694-a00d-7a4cd5d48e1e None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:24.566106 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:24.573009 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc531bc-e9f4-4b21-9235-6d7e1c1d636b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:24.574678 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc531bc-e9f4-4b21-9235-6d7e1c1d636b None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:24.574678 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc531bc-e9f4-4b21-9235-6d7e1c1d636b None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:24.574678 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc531bc-e9f4-4b21-9235-6d7e1c1d636b None admin] PATH_INFO: `/v3/users` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:24.575035 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc531bc-e9f4-4b21-9235-6d7e1c1d636b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:24.575388 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc531bc-e9f4-4b21-9235-6d7e1c1d636b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:24.576878 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc531bc-e9f4-4b21-9235-6d7e1c1d636b None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:24.595716 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 343/685] 10.222.0.18 () {56 vars in 1135 bytes} [Wed Aug 24 17:08:24 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:24.596641 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc531bc-e9f4-4b21-9235-6d7e1c1d636b None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:24.641437 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:24.647488 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3ade3a-e711-4020-91d4-968034b035c4 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:24.686153 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15998bfa-7d6f-4982-9026-fed47f242da8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:24.686667 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15998bfa-7d6f-4982-9026-fed47f242da8 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:24.687019 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15998bfa-7d6f-4982-9026-fed47f242da8 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:24.687246 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15998bfa-7d6f-4982-9026-fed47f242da8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:24.687950 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15998bfa-7d6f-4982-9026-fed47f242da8 None admin] RBAC: Adding query filter params (role.id=e7db2a847f56438bac13d0ff21c96b73, scope.project.id=46ab158bc3644c888557781b0630b0b8, user.id=be06246d47584b529150ceb98a5cfd95) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:24.688255 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15998bfa-7d6f-4982-9026-fed47f242da8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:24.689769 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15998bfa-7d6f-4982-9026-fed47f242da8 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:24.703086 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 343/686] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 24 17:08:24 2022] GET /identity/v3/role_assignments?user.id=be06246d47584b529150ceb98a5cfd95&scope.project.id=46ab158bc3644c888557781b0630b0b8&role.id=e7db2a847f56438bac13d0ff21c96b73 => generated 608 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:08:24.703086 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15998bfa-7d6f-4982-9026-fed47f242da8 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:24.762926 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc531bc-e9f4-4b21-9235-6d7e1c1d636b None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:24.856110 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2289a536-55bc-408f-8587-bb950fbc54ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:24.857036 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2289a536-55bc-408f-8587-bb950fbc54ac None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:24.857457 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2289a536-55bc-408f-8587-bb950fbc54ac None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:24.857868 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2289a536-55bc-408f-8587-bb950fbc54ac None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:24.860786 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15998bfa-7d6f-4982-9026-fed47f242da8 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:24.891254 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2289a536-55bc-408f-8587-bb950fbc54ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:08:24.896444 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-2289a536-55bc-408f-8587-bb950fbc54ac None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:08:24.896444 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:24.896444 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:24.896444 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:24.896444 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:24.896444 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:24.896444 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:24.896444 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:24.896444 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:24.896444 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:24.896444 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:24.896444 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:24.896444 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:08:24.896444 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:08:24.896444 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:08:24.896444 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:08:24.897376 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:08:24.897376 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:08:24.897376 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:08:24.897376 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:08:24.897376 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:24.897376 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:24.897376 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:24.897376 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:24.897376 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:24.897376 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:24.897376 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:24.897376 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:24.897376 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:24.897376 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:24.897376 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:24.897376 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:24.898213 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:24.898213 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:24.898213 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:24.898213 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:24.898213 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:08:24.898213 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:08:24.898213 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:08:24.898213 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:08:24.898213 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:08:24.898213 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:08:24.898213 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:08:24.898213 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:08:24.903800 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b9e1d9-2962-4067-8cab-57862a55e5ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:24.904266 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b9e1d9-2962-4067-8cab-57862a55e5ea None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:24.904502 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b9e1d9-2962-4067-8cab-57862a55e5ea None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:24.904996 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b9e1d9-2962-4067-8cab-57862a55e5ea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:24.905201 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 344/687] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Aug 24 17:08:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:24.908238 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2289a536-55bc-408f-8587-bb950fbc54ac None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:24.946453 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:24.967245 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:25.004262 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b9e1d9-2962-4067-8cab-57862a55e5ea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:25.005866 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b9e1d9-2962-4067-8cab-57862a55e5ea None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:25.020056 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2289a536-55bc-408f-8587-bb950fbc54ac None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:25.058266 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744b8cbe-68af-4267-832d-6548f597a646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:25.059840 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744b8cbe-68af-4267-832d-6548f597a646 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:25.060339 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744b8cbe-68af-4267-832d-6548f597a646 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:25.060798 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744b8cbe-68af-4267-832d-6548f597a646 None admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:25.061755 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744b8cbe-68af-4267-832d-6548f597a646 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:25.062305 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744b8cbe-68af-4267-832d-6548f597a646 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:25.064123 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744b8cbe-68af-4267-832d-6548f597a646 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:25.074165 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 345/688] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 24 17:08:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:25.080005 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744b8cbe-68af-4267-832d-6548f597a646 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:25.142475 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 344/689] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 24 17:08:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 17:08:25.148283 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b9e1d9-2962-4067-8cab-57862a55e5ea None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:25.227387 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:25.237310 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744b8cbe-68af-4267-832d-6548f597a646 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:25.270712 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:25.306802 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b9e1d9-2962-4067-8cab-57862a55e5ea None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:25.308817 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3746758-eb24-4e8f-bf5d-90ac83f208f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:25.309486 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3746758-eb24-4e8f-bf5d-90ac83f208f0 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:25.309766 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3746758-eb24-4e8f-bf5d-90ac83f208f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:25.310045 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3746758-eb24-4e8f-bf5d-90ac83f208f0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:25.329384 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3746758-eb24-4e8f-bf5d-90ac83f208f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:25.330849 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3746758-eb24-4e8f-bf5d-90ac83f208f0 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:25.346490 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-e3746758-eb24-4e8f-bf5d-90ac83f208f0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:08:25.346490 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:25.346490 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:25.346490 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:25.346490 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:25.346490 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:25.346490 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:25.346490 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:25.346490 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:25.346490 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:25.346490 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:25.346490 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:25.346490 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:08:25.346490 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:08:25.346490 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:08:25.346490 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:08:25.347482 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:25.347482 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:25.347482 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:25.347482 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:25.347482 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:25.347482 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:25.347482 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:25.347482 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:25.347482 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:25.347482 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:25.347482 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:25.347482 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:25.347482 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:25.347482 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:25.347482 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:25.347482 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:25.347482 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:08:25.348380 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:08:25.348380 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:08:25.348380 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:08:25.348380 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 346/690] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 24 17:08:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:08:25.354643 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3746758-eb24-4e8f-bf5d-90ac83f208f0 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:25.390705 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90152a26-b4fe-4792-b6e5-c09b1798c984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:25.391375 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90152a26-b4fe-4792-b6e5-c09b1798c984 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:25.391654 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90152a26-b4fe-4792-b6e5-c09b1798c984 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:25.391931 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90152a26-b4fe-4792-b6e5-c09b1798c984 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:25.411496 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90152a26-b4fe-4792-b6e5-c09b1798c984 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:25.412970 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90152a26-b4fe-4792-b6e5-c09b1798c984 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:25.442065 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-90152a26-b4fe-4792-b6e5-c09b1798c984 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:08:25.442065 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:25.442065 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:25.442065 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:25.442065 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:25.442065 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:25.442065 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:25.442065 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:25.442065 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:25.442065 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:25.442065 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:25.442065 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:25.442065 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:08:25.442065 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:08:25.442065 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:08:25.442065 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:08:25.443103 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:25.443103 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:25.443103 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:25.443103 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:25.443103 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:25.443103 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:25.443103 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:25.443103 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:25.443103 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:25.443103 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:25.443103 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:25.443103 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:25.443103 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:25.443103 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:25.443103 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:25.443103 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:25.443103 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:08:25.444038 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:08:25.444038 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:08:25.444038 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:08:25.455562 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 345/691] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 24 17:08:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:25.456454 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90152a26-b4fe-4792-b6e5-c09b1798c984 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:25.542551 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3746758-eb24-4e8f-bf5d-90ac83f208f0 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:25.582434 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90152a26-b4fe-4792-b6e5-c09b1798c984 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:25.597879 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:25.611023 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0601c683-f53c-4791-83bf-082b68ada653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:25.611640 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0601c683-f53c-4791-83bf-082b68ada653 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:25.611946 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0601c683-f53c-4791-83bf-082b68ada653 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:25.612219 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0601c683-f53c-4791-83bf-082b68ada653 None admin] PATH_INFO: `/v3/domains` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:25.612859 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0601c683-f53c-4791-83bf-082b68ada653 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:25.613159 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0601c683-f53c-4791-83bf-082b68ada653 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:25.614543 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0601c683-f53c-4791-83bf-082b68ada653 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:25.637910 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 347/692] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 24 17:08:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:08:25.639502 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c3cd23-07d9-43af-ad36-a94baf8c47fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:25.640116 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c3cd23-07d9-43af-ad36-a94baf8c47fe None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:25.640390 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c3cd23-07d9-43af-ad36-a94baf8c47fe None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:25.640654 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c3cd23-07d9-43af-ad36-a94baf8c47fe None admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:25.641300 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c3cd23-07d9-43af-ad36-a94baf8c47fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:25.641598 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c3cd23-07d9-43af-ad36-a94baf8c47fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:25.643038 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c3cd23-07d9-43af-ad36-a94baf8c47fe None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:25.645516 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0601c683-f53c-4791-83bf-082b68ada653 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:25.669015 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 346/693] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 24 17:08:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:25.673275 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c3cd23-07d9-43af-ad36-a94baf8c47fe None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:25.695249 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:25.793980 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0601c683-f53c-4791-83bf-082b68ada653 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:25.847062 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c3cd23-07d9-43af-ad36-a94baf8c47fe None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:25.861535 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ae537b-a9b5-42ed-a2e3-6a34faade8a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:25.862179 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ae537b-a9b5-42ed-a2e3-6a34faade8a0 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:25.862455 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ae537b-a9b5-42ed-a2e3-6a34faade8a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:25.862726 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ae537b-a9b5-42ed-a2e3-6a34faade8a0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:25.876351 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ae537b-a9b5-42ed-a2e3-6a34faade8a0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:25.877926 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ae537b-a9b5-42ed-a2e3-6a34faade8a0 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:25.888047 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecdb67e-057e-4c2b-97e4-8c6073bcacb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:25.888681 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecdb67e-057e-4c2b-97e4-8c6073bcacb1 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:25.888959 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecdb67e-057e-4c2b-97e4-8c6073bcacb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:25.889229 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecdb67e-057e-4c2b-97e4-8c6073bcacb1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:25.901206 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-72ae537b-a9b5-42ed-a2e3-6a34faade8a0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 17:08:25.901206 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:25.901206 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:25.901206 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:25.901206 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:25.901206 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:25.901206 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:25.901206 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:25.901206 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:25.901206 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:25.901206 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:25.901206 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:25.901206 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:08:25.901206 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:08:25.901206 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:08:25.901206 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:08:25.902169 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:25.902169 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:25.902169 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:08:25.902169 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:08:25.902169 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:08:25.902169 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:08:25.902169 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:25.902169 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:25.902169 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:25.902169 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:25.902169 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:25.902169 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:25.902169 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:25.902169 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:25.902169 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:25.902169 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:25.902169 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:25.902169 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:25.903745 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:25.903745 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:25.903745 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:08:25.903745 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:08:25.903745 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:08:25.903745 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:08:25.903745 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:08:25.903745 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:08:25.903745 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 17:08:25.903745 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:08:25.903745 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 348/694] 10.222.0.18 () {56 vars in 1093 bytes} [Wed Aug 24 17:08:25 2022] GET /identity/v3/users/nova => generated 82 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:25.904337 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecdb67e-057e-4c2b-97e4-8c6073bcacb1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:25.909812 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecdb67e-057e-4c2b-97e4-8c6073bcacb1 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:25.911238 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ae537b-a9b5-42ed-a2e3-6a34faade8a0 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:25.935918 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-1ecdb67e-057e-4c2b-97e4-8c6073bcacb1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:08:25.935918 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:25.935918 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:25.935918 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:25.935918 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:25.935918 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:25.935918 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:25.935918 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:25.935918 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:25.935918 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:25.935918 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:25.935918 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:25.935918 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:08:25.935918 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:08:25.935918 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:08:25.937448 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:08:25.937448 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:25.937448 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:25.937448 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:25.937448 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:25.937448 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:25.937448 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:25.937448 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:25.937448 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:25.937448 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:25.937448 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:25.937448 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:25.937448 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:25.937448 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:25.937448 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:25.937448 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:25.937448 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:25.939549 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:08:25.939549 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:08:25.939549 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:08:25.939549 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:08:25.939549 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:08:25.939549 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:08:25.939549 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:08:25.939549 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:08:25.939549 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 347/695] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 24 17:08:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:08:25.947851 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecdb67e-057e-4c2b-97e4-8c6073bcacb1 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:26.036954 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:26.037209 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:26.041244 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ae537b-a9b5-42ed-a2e3-6a34faade8a0 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:26.076584 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a134268-4404-4cf8-baac-f069687794dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:26.077470 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a134268-4404-4cf8-baac-f069687794dd None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:26.077912 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a134268-4404-4cf8-baac-f069687794dd None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:26.078333 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a134268-4404-4cf8-baac-f069687794dd None admin] PATH_INFO: `/v3/users` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:26.079260 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a134268-4404-4cf8-baac-f069687794dd None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:26.079702 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a134268-4404-4cf8-baac-f069687794dd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:26.081286 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a134268-4404-4cf8-baac-f069687794dd None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:26.103220 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 349/696] 10.222.0.18 () {56 vars in 1133 bytes} [Wed Aug 24 17:08:25 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:26.105699 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a134268-4404-4cf8-baac-f069687794dd None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:26.187261 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecdb67e-057e-4c2b-97e4-8c6073bcacb1 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:26.212451 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a134268-4404-4cf8-baac-f069687794dd None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:26.223765 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c43691c-3168-4263-a216-9daea9886002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:26.224635 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c43691c-3168-4263-a216-9daea9886002 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:26.225065 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c43691c-3168-4263-a216-9daea9886002 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:26.225487 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c43691c-3168-4263-a216-9daea9886002 None admin] PATH_INFO: `/v3/projects` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:26.226428 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c43691c-3168-4263-a216-9daea9886002 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:26.226954 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c43691c-3168-4263-a216-9daea9886002 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:26.228532 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c43691c-3168-4263-a216-9daea9886002 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:26.247840 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 348/697] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Aug 24 17:08:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:26.251184 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645b56bc-e4b4-46e2-a2be-289d4e317c69 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:26.251869 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645b56bc-e4b4-46e2-a2be-289d4e317c69 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:26.252126 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9193cea5-20e6-4038-81a5-8ffaeb7d2ead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:26.252649 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645b56bc-e4b4-46e2-a2be-289d4e317c69 None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:26.252988 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9193cea5-20e6-4038-81a5-8ffaeb7d2ead None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:26.253790 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9193cea5-20e6-4038-81a5-8ffaeb7d2ead None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:26.253920 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9193cea5-20e6-4038-81a5-8ffaeb7d2ead None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:26.255700 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 349/698] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:08:26 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:08:26.257165 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a33fc05-0cfc-4ecb-b78e-56a56a01bb3a None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:26.257599 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a33fc05-0cfc-4ecb-b78e-56a56a01bb3a None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:26.258019 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a33fc05-0cfc-4ecb-b78e-56a56a01bb3a None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:26.259766 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 350/699] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:08:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:08:26.263331 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a33fc05-0cfc-4ecb-b78e-56a56a01bb3a None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:26.269516 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:26.332776 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9193cea5-20e6-4038-81a5-8ffaeb7d2ead None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:26.334136 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9193cea5-20e6-4038-81a5-8ffaeb7d2ead None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:26.367698 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:26.377805 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a33fc05-0cfc-4ecb-b78e-56a56a01bb3a None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:26.417009 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450b5609-cba8-424a-a041-9f286a896999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:26.417935 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450b5609-cba8-424a-a041-9f286a896999 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:26.418206 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450b5609-cba8-424a-a041-9f286a896999 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:26.419311 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450b5609-cba8-424a-a041-9f286a896999 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:26.420048 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450b5609-cba8-424a-a041-9f286a896999 None admin] RBAC: Adding query filter params (role.id=e7db2a847f56438bac13d0ff21c96b73, scope.project.id=46ab158bc3644c888557781b0630b0b8, user.id=ef8caa353be34499a58cd25b72ee4032) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:26.420359 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450b5609-cba8-424a-a041-9f286a896999 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:26.422485 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450b5609-cba8-424a-a041-9f286a896999 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:26.430698 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 351/700] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 24 17:08:26 2022] GET /identity/v3/role_assignments?user.id=ef8caa353be34499a58cd25b72ee4032&scope.project.id=46ab158bc3644c888557781b0630b0b8&role.id=e7db2a847f56438bac13d0ff21c96b73 => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:26.432009 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2219ab3a-4692-46e6-8aab-26388a5b42cd None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:26.432361 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2219ab3a-4692-46e6-8aab-26388a5b42cd None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:26.432700 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2219ab3a-4692-46e6-8aab-26388a5b42cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:26.450674 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 350/701] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 24 17:08:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 17:08:26.452475 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bbbbd6-dde2-43d6-bfc2-3f4c7d62f317 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:26.452584 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-2219ab3a-4692-46e6-8aab-26388a5b42cd None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:08:26.453026 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bbbbd6-dde2-43d6-bfc2-3f4c7d62f317 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:26.453483 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bbbbd6-dde2-43d6-bfc2-3f4c7d62f317 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:26.453924 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-2219ab3a-4692-46e6-8aab-26388a5b42cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:08:26.475437 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-97bbbbd6-dde2-43d6-bfc2-3f4c7d62f317 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:08:26.477655 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-97bbbbd6-dde2-43d6-bfc2-3f4c7d62f317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:08:26.531155 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:26.539180 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 352/702] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:08:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:08:26.539180 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2219ab3a-4692-46e6-8aab-26388a5b42cd None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:26.590106 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 351/703] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:08:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:08:26.667233 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:26.682965 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2219ab3a-4692-46e6-8aab-26388a5b42cd None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:26.719721 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d79ca9b-b553-4bb9-b79f-39884b4a7687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:26.720355 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d79ca9b-b553-4bb9-b79f-39884b4a7687 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:26.720635 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d79ca9b-b553-4bb9-b79f-39884b4a7687 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:26.720907 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d79ca9b-b553-4bb9-b79f-39884b4a7687 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:26.731943 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d79ca9b-b553-4bb9-b79f-39884b4a7687 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:26.733484 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d79ca9b-b553-4bb9-b79f-39884b4a7687 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:26.750641 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-9d79ca9b-b553-4bb9-b79f-39884b4a7687 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:08:26.750641 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:26.750641 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:26.750641 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:26.750641 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:26.750641 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:26.750641 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:26.750641 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:26.750641 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:26.750641 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:26.750641 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:26.750641 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:26.750641 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:08:26.750641 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:08:26.750641 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:08:26.750641 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:08:26.751951 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:26.751951 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:26.751951 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:26.751951 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:26.751951 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:26.751951 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:26.751951 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:26.751951 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:26.751951 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:26.751951 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:26.751951 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:26.751951 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:26.751951 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:26.751951 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:26.751951 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:26.751951 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:26.751951 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:08:26.753215 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:08:26.753215 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:08:26.753215 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:08:26.755538 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 353/704] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 24 17:08:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:26.759990 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bbbbd6-dde2-43d6-bfc2-3f4c7d62f317 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:26.867983 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d79ca9b-b553-4bb9-b79f-39884b4a7687 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:26.871281 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bbbbd6-dde2-43d6-bfc2-3f4c7d62f317 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:26.920287 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f4e1b3-d68c-4c0c-b312-212693033ecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:26.920909 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f4e1b3-d68c-4c0c-b312-212693033ecd None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:26.921462 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f4e1b3-d68c-4c0c-b312-212693033ecd None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:26.921652 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f4e1b3-d68c-4c0c-b312-212693033ecd None admin] PATH_INFO: `/v3/domains` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:26.922265 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f4e1b3-d68c-4c0c-b312-212693033ecd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:26.922571 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f4e1b3-d68c-4c0c-b312-212693033ecd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:26.924086 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f4e1b3-d68c-4c0c-b312-212693033ecd None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:26.936600 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 352/705] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 24 17:08:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:26.937423 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f4e1b3-d68c-4c0c-b312-212693033ecd None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:26.977357 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:26.988099 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d79ca9b-b553-4bb9-b79f-39884b4a7687 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:27.029608 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4471f1f8-bc05-4f59-bf5b-b00222e39ed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:27.030295 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4471f1f8-bc05-4f59-bf5b-b00222e39ed4 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:27.030570 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4471f1f8-bc05-4f59-bf5b-b00222e39ed4 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:27.030876 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4471f1f8-bc05-4f59-bf5b-b00222e39ed4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:27.039062 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:27.045082 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4471f1f8-bc05-4f59-bf5b-b00222e39ed4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:27.047349 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4471f1f8-bc05-4f59-bf5b-b00222e39ed4 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:27.067261 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-4471f1f8-bc05-4f59-bf5b-b00222e39ed4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:08:27.067261 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:27.067261 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:27.067261 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:27.067261 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:27.067261 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:27.067261 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:27.067261 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:27.067261 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:27.067261 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:27.067261 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:27.067261 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:27.067261 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:08:27.067261 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:08:27.067261 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:08:27.067261 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:08:27.068658 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:27.068658 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:27.068658 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:27.068658 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:27.068658 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:27.068658 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:27.068658 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:27.068658 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:27.068658 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:27.068658 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:27.068658 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:27.068658 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:27.068658 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:27.068658 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:27.068658 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:27.068658 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:27.068658 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:08:27.069828 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:08:27.069828 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:08:27.069828 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:08:27.070049 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f4e1b3-d68c-4c0c-b312-212693033ecd None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:27.074929 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 354/706] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 24 17:08:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:08:27.074929 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4471f1f8-bc05-4f59-bf5b-b00222e39ed4 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:27.110310 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c7f9ab-9593-4e5b-b111-c0d21afc2b10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:27.111116 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c7f9ab-9593-4e5b-b111-c0d21afc2b10 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:27.111442 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c7f9ab-9593-4e5b-b111-c0d21afc2b10 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:27.111721 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c7f9ab-9593-4e5b-b111-c0d21afc2b10 None admin] PATH_INFO: `/v3/services/network` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:27.112302 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c7f9ab-9593-4e5b-b111-c0d21afc2b10 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:27.113670 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c7f9ab-9593-4e5b-b111-c0d21afc2b10 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:27.126786 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-37c7f9ab-9593-4e5b-b111-c0d21afc2b10 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 17:08:27.126786 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:27.126786 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 17:08:27.126786 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 17:08:27.126786 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 17:08:27.126786 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 17:08:27.126786 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 17:08:27.126786 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:08:27.126786 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 17:08:27.126786 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:08:27.126786 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 17:08:27.126786 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:08:27.126786 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:27.126786 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:27.126786 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:27.126786 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:27.126786 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:27.129138 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:27.129138 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:27.129138 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:27.129138 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:27.129138 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:27.129138 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:27.129138 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 17:08:27.129138 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 17:08:27.129138 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 17:08:27.129138 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 17:08:27.129138 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:27.129138 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:27.129138 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:27.129138 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:27.129138 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:27.129138 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:27.131288 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:27.131288 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:27.131288 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:27.131288 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:27.131288 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:27.131288 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:27.131288 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:27.131288 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:27.131288 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:27.131288 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:27.131288 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 17:08:27.131288 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:08:27.131288 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 17:08:27.131288 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:08:27.131288 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 353/707] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 24 17:08:26 2022] GET /identity/v3/services/network => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:27.133297 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c7f9ab-9593-4e5b-b111-c0d21afc2b10 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:27.206050 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4471f1f8-bc05-4f59-bf5b-b00222e39ed4 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:27.235277 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:27.242859 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8be477-473c-464f-b5ab-d5dd8083e281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:27.243539 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8be477-473c-464f-b5ab-d5dd8083e281 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:27.243821 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8be477-473c-464f-b5ab-d5dd8083e281 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:27.248748 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c7f9ab-9593-4e5b-b111-c0d21afc2b10 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:27.248872 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8be477-473c-464f-b5ab-d5dd8083e281 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:27.255202 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8be477-473c-464f-b5ab-d5dd8083e281 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:27.256694 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8be477-473c-464f-b5ab-d5dd8083e281 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:27.268871 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-7a8be477-473c-464f-b5ab-d5dd8083e281 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:08:27.268871 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:27.268871 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:27.268871 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:27.268871 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:27.268871 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:27.268871 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:27.268871 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:27.268871 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:27.268871 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:27.268871 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:27.268871 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:27.268871 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:08:27.268871 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:08:27.268871 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:08:27.270831 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:08:27.270831 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:27.270831 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:27.270831 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:27.270831 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:27.270831 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:27.270831 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:27.270831 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:27.270831 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:27.270831 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:27.270831 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:27.270831 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:27.270831 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:27.270831 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:27.270831 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:27.270831 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:27.270831 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:27.272987 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:08:27.272987 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:08:27.272987 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:08:27.272987 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:08:27.272987 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:08:27.272987 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:08:27.272987 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:08:27.272987 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:08:27.272987 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 355/708] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 24 17:08:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:27.272987 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8be477-473c-464f-b5ab-d5dd8083e281 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:27.287273 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3420191a-8a98-4981-950c-691686200cbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:27.287889 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3420191a-8a98-4981-950c-691686200cbf None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:27.288170 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3420191a-8a98-4981-950c-691686200cbf None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:27.288445 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3420191a-8a98-4981-950c-691686200cbf None admin] PATH_INFO: `/v3/domains` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:27.289097 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3420191a-8a98-4981-950c-691686200cbf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:27.289398 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3420191a-8a98-4981-950c-691686200cbf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:27.290776 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3420191a-8a98-4981-950c-691686200cbf None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:27.307163 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 354/709] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 24 17:08:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:27.307163 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be184aad-9985-43bc-a5f6-3c96c87fb6c3 None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:27.307163 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be184aad-9985-43bc-a5f6-3c96c87fb6c3 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:27.307163 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be184aad-9985-43bc-a5f6-3c96c87fb6c3 None None] PATH_INFO: `/` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:27.307163 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 355/710] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:08:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:08:27.307163 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be184aad-9985-43bc-a5f6-3c96c87fb6c3 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:27.330489 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:27.381728 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8be477-473c-464f-b5ab-d5dd8083e281 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:27.419112 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50f81d4-055b-48b7-afc1-3d68ff610bb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:27.419767 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50f81d4-055b-48b7-afc1-3d68ff610bb8 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:27.420051 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50f81d4-055b-48b7-afc1-3d68ff610bb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:27.420322 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50f81d4-055b-48b7-afc1-3d68ff610bb8 None admin] PATH_INFO: `/v3/services` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:27.420953 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50f81d4-055b-48b7-afc1-3d68ff610bb8 None admin] RBAC: Adding query filter params (name=network) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:27.421256 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50f81d4-055b-48b7-afc1-3d68ff610bb8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:27.422610 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50f81d4-055b-48b7-afc1-3d68ff610bb8 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:27.424058 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be184aad-9985-43bc-a5f6-3c96c87fb6c3 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:27.427429 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 356/711] 10.222.0.18 () {56 vars in 1112 bytes} [Wed Aug 24 17:08:27 2022] GET /identity/v3/services?name=network => generated 124 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:27.432350 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50f81d4-055b-48b7-afc1-3d68ff610bb8 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:27.461675 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4554fc5-7595-4e55-a809-67c1267fc1d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:27.462536 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4554fc5-7595-4e55-a809-67c1267fc1d7 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:27.462992 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4554fc5-7595-4e55-a809-67c1267fc1d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:27.463430 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4554fc5-7595-4e55-a809-67c1267fc1d7 None admin] PATH_INFO: `/v3/projects` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:27.464298 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4554fc5-7595-4e55-a809-67c1267fc1d7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:27.464756 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4554fc5-7595-4e55-a809-67c1267fc1d7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:27.466872 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4554fc5-7595-4e55-a809-67c1267fc1d7 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:27.484075 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 356/712] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Aug 24 17:08:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:27.485687 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af221720-d9dd-4689-82e4-22bbfff1a24f None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:27.486142 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af221720-d9dd-4689-82e4-22bbfff1a24f None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:27.486418 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af221720-d9dd-4689-82e4-22bbfff1a24f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:27.504916 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-af221720-d9dd-4689-82e4-22bbfff1a24f None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:08:27.506097 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-af221720-d9dd-4689-82e4-22bbfff1a24f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:08:27.518310 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:27.536745 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50f81d4-055b-48b7-afc1-3d68ff610bb8 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:27.557814 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:27.578548 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcff1c86-0da3-49e9-82a2-e27985d166d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:27.579184 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcff1c86-0da3-49e9-82a2-e27985d166d8 None admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:27.579465 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcff1c86-0da3-49e9-82a2-e27985d166d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:27.579735 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcff1c86-0da3-49e9-82a2-e27985d166d8 None admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:27.580378 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcff1c86-0da3-49e9-82a2-e27985d166d8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:27.581794 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcff1c86-0da3-49e9-82a2-e27985d166d8 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:27.582608 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 357/713] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:08:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:08:27.583617 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af221720-d9dd-4689-82e4-22bbfff1a24f None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:27.677106 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 357/714] 10.222.0.18 () {60 vars in 1132 bytes} [Wed Aug 24 17:08:27 2022] POST /identity/v3/users => generated 254 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:08:27.677106 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcff1c86-0da3-49e9-82a2-e27985d166d8 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:27.729678 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af221720-d9dd-4689-82e4-22bbfff1a24f None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:27.799635 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf86d709-692c-4dcd-a439-a01a45ec61aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:27.800309 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf86d709-692c-4dcd-a439-a01a45ec61aa None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:27.800589 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf86d709-692c-4dcd-a439-a01a45ec61aa None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:27.800856 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf86d709-692c-4dcd-a439-a01a45ec61aa None admin] PATH_INFO: `/v3/services` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:27.801580 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf86d709-692c-4dcd-a439-a01a45ec61aa None admin] RBAC: Adding query filter params (type=network) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:27.801887 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf86d709-692c-4dcd-a439-a01a45ec61aa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:27.803315 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf86d709-692c-4dcd-a439-a01a45ec61aa None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:27.817673 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 358/715] 10.222.0.18 () {56 vars in 1112 bytes} [Wed Aug 24 17:08:27 2022] GET /identity/v3/services?type=network => generated 124 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:27.870632 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcff1c86-0da3-49e9-82a2-e27985d166d8 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:27.882630 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf86d709-692c-4dcd-a439-a01a45ec61aa None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:27.900733 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:27.922265 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0721a02-c62e-400a-af51-b99c47821524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:27.923421 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0721a02-c62e-400a-af51-b99c47821524 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:27.923878 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0721a02-c62e-400a-af51-b99c47821524 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:27.924582 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0721a02-c62e-400a-af51-b99c47821524 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:27.925932 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0721a02-c62e-400a-af51-b99c47821524 None admin] RBAC: Adding query filter params (role.id=c5346bc54abf445f9900edf94264c850, scope.project.id=46ab158bc3644c888557781b0630b0b8, user.id=557aca9255cb4bbb8af52e796c5c5177) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:27.926402 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0721a02-c62e-400a-af51-b99c47821524 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:27.928430 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0721a02-c62e-400a-af51-b99c47821524 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:27.943534 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 358/716] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 24 17:08:27 2022] GET /identity/v3/role_assignments?user.id=557aca9255cb4bbb8af52e796c5c5177&scope.project.id=46ab158bc3644c888557781b0630b0b8&role.id=c5346bc54abf445f9900edf94264c850 => generated 259 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:28.023154 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf86d709-692c-4dcd-a439-a01a45ec61aa None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:28.061155 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50a90d2-1b5c-4201-bdcd-71744414cfee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:28.061985 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50a90d2-1b5c-4201-bdcd-71744414cfee None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:28.062418 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50a90d2-1b5c-4201-bdcd-71744414cfee None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:28.062790 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50a90d2-1b5c-4201-bdcd-71744414cfee None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:28.063556 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50a90d2-1b5c-4201-bdcd-71744414cfee None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:28.065043 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50a90d2-1b5c-4201-bdcd-71744414cfee None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:28.074768 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-d50a90d2-1b5c-4201-bdcd-71744414cfee None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 17:08:28.074768 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:28.074768 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 17:08:28.074768 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 17:08:28.074768 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 17:08:28.074768 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 17:08:28.074768 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 17:08:28.074768 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:08:28.074768 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 17:08:28.074768 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:08:28.074768 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 17:08:28.074768 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:08:28.074768 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:28.074768 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:28.074768 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:28.074768 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:28.074768 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:28.076103 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:28.076103 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:28.076103 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:28.076103 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:28.076103 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:28.076103 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:28.076103 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 17:08:28.076103 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 17:08:28.076103 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 17:08:28.076103 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 17:08:28.076103 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:28.076103 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:28.076103 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:28.076103 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:28.076103 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:28.076103 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:28.077175 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:28.077175 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:28.077175 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:28.077175 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:28.077175 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:28.077175 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:28.077175 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:28.077175 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:28.077175 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:28.077175 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:28.077175 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 17:08:28.077175 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:08:28.077175 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 17:08:28.077175 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:08:28.079492 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 359/717] 10.222.0.18 () {56 vars in 1129 bytes} [Wed Aug 24 17:08:27 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:28.085195 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0721a02-c62e-400a-af51-b99c47821524 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:28.125410 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:28.195697 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0721a02-c62e-400a-af51-b99c47821524 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:28.231823 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fd8e5a-87e5-47c5-a952-e951e1d7fec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:28.232746 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fd8e5a-87e5-47c5-a952-e951e1d7fec8 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:28.233128 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fd8e5a-87e5-47c5-a952-e951e1d7fec8 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:28.233504 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fd8e5a-87e5-47c5-a952-e951e1d7fec8 None admin] PATH_INFO: `/v3/services` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:28.234234 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fd8e5a-87e5-47c5-a952-e951e1d7fec8 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:28.234637 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fd8e5a-87e5-47c5-a952-e951e1d7fec8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:28.236129 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fd8e5a-87e5-47c5-a952-e951e1d7fec8 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:28.241240 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 359/718] 10.222.0.18 () {56 vars in 1124 bytes} [Wed Aug 24 17:08:28 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:28.247561 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50a90d2-1b5c-4201-bdcd-71744414cfee None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:28.344278 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:28.381071 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50a90d2-1b5c-4201-bdcd-71744414cfee None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:28.452684 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4131f3d-86b1-4149-9211-574ea1c00a56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:28.453289 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4131f3d-86b1-4149-9211-574ea1c00a56 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:28.453516 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4131f3d-86b1-4149-9211-574ea1c00a56 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:28.453746 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4131f3d-86b1-4149-9211-574ea1c00a56 None admin] PATH_INFO: `/v3/services` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:28.454376 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4131f3d-86b1-4149-9211-574ea1c00a56 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:28.454676 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4131f3d-86b1-4149-9211-574ea1c00a56 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:28.456077 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4131f3d-86b1-4149-9211-574ea1c00a56 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:28.467674 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 360/719] 10.222.0.18 () {56 vars in 1124 bytes} [Wed Aug 24 17:08:28 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:28.748682 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00eb3eb-9008-4c51-9908-c58b46c87e75 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:28.748896 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00eb3eb-9008-4c51-9908-c58b46c87e75 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:28.749163 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00eb3eb-9008-4c51-9908-c58b46c87e75 None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:28.750618 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 360/720] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:08:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:08:28.758944 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029d1a8a-3870-4f71-bd74-baa5fe164928 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:28.758944 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029d1a8a-3870-4f71-bd74-baa5fe164928 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:28.758944 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029d1a8a-3870-4f71-bd74-baa5fe164928 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:28.776550 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-029d1a8a-3870-4f71-bd74-baa5fe164928 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:08:28.778709 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-029d1a8a-3870-4f71-bd74-baa5fe164928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:08:28.852586 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 361/721] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:08:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:08:29.204769 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00eb3eb-9008-4c51-9908-c58b46c87e75 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:29.318678 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:29.328176 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00eb3eb-9008-4c51-9908-c58b46c87e75 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:29.370422 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156b6c0f-0b34-443d-89c8-e9165ff5d458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:29.371117 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156b6c0f-0b34-443d-89c8-e9165ff5d458 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:29.376462 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156b6c0f-0b34-443d-89c8-e9165ff5d458 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:29.376462 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156b6c0f-0b34-443d-89c8-e9165ff5d458 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:29.385865 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156b6c0f-0b34-443d-89c8-e9165ff5d458 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:08:29.391555 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-156b6c0f-0b34-443d-89c8-e9165ff5d458 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:08:29.391555 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:29.391555 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:29.391555 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:29.391555 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:29.391555 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:29.391555 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:29.391555 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:29.391555 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:29.391555 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:29.391555 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:29.391555 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:29.391555 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:08:29.391555 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:08:29.391555 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:08:29.391555 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:08:29.392688 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:08:29.392688 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:08:29.392688 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:08:29.392688 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:08:29.392688 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:29.392688 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:29.392688 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:29.392688 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:29.392688 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:29.392688 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:29.392688 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:29.392688 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:29.392688 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:29.392688 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:29.392688 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:29.392688 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:29.393569 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:29.393569 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:29.393569 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:29.393569 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:29.393569 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:08:29.393569 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:08:29.393569 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:08:29.393569 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:08:29.393569 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:08:29.393569 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:08:29.393569 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:08:29.393569 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:08:29.393569 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 361/722] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Aug 24 17:08:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:29.398609 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029d1a8a-3870-4f71-bd74-baa5fe164928 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:29.484976 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:29.528495 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029d1a8a-3870-4f71-bd74-baa5fe164928 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:29.583323 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01653411-e79f-4fe0-bb88-3771dfc08fb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:29.583933 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01653411-e79f-4fe0-bb88-3771dfc08fb7 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:29.584207 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01653411-e79f-4fe0-bb88-3771dfc08fb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:29.584477 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01653411-e79f-4fe0-bb88-3771dfc08fb7 None admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:29.585136 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01653411-e79f-4fe0-bb88-3771dfc08fb7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:29.585431 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01653411-e79f-4fe0-bb88-3771dfc08fb7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:29.586779 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01653411-e79f-4fe0-bb88-3771dfc08fb7 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:29.598183 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 362/723] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 24 17:08:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:29.603435 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156b6c0f-0b34-443d-89c8-e9165ff5d458 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:29.723576 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156b6c0f-0b34-443d-89c8-e9165ff5d458 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:29.758446 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7b822e-1be8-4c0a-89aa-bc5fd933f131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:29.759020 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7b822e-1be8-4c0a-89aa-bc5fd933f131 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:29.759299 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7b822e-1be8-4c0a-89aa-bc5fd933f131 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:29.759568 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7b822e-1be8-4c0a-89aa-bc5fd933f131 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:29.770232 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7b822e-1be8-4c0a-89aa-bc5fd933f131 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:29.771695 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7b822e-1be8-4c0a-89aa-bc5fd933f131 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:29.786161 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-6f7b822e-1be8-4c0a-89aa-bc5fd933f131 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:08:29.786161 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:29.786161 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:29.786161 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:29.786161 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:29.786161 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:29.786161 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:29.786161 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:29.786161 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:29.786161 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:29.786161 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:29.786161 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:29.786161 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:08:29.786161 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:08:29.786161 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:08:29.786161 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:08:29.787162 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:29.787162 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:29.787162 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:29.787162 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:29.787162 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:29.787162 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:29.787162 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:29.787162 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:29.787162 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:29.787162 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:29.787162 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:29.787162 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:29.787162 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:29.787162 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:29.787162 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:29.787162 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:29.787162 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:08:29.788676 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:08:29.788676 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:08:29.788676 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:08:29.788676 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 362/724] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 24 17:08:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:29.793070 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01653411-e79f-4fe0-bb88-3771dfc08fb7 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:29.927479 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01653411-e79f-4fe0-bb88-3771dfc08fb7 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:29.968186 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:29.974459 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc05c617-f9b9-46d8-af6f-e79fe454da29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:29.975099 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc05c617-f9b9-46d8-af6f-e79fe454da29 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:29.975379 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc05c617-f9b9-46d8-af6f-e79fe454da29 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:29.975651 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc05c617-f9b9-46d8-af6f-e79fe454da29 None admin] PATH_INFO: `/v3/domains` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:29.976289 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc05c617-f9b9-46d8-af6f-e79fe454da29 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:29.976585 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc05c617-f9b9-46d8-af6f-e79fe454da29 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:29.977947 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc05c617-f9b9-46d8-af6f-e79fe454da29 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:29.993758 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 363/725] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 24 17:08:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:29.999264 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7b822e-1be8-4c0a-89aa-bc5fd933f131 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:30.012726 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:30.066849 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b6320e-8e29-414e-9e92-17affdd9160e None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:30.067229 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b6320e-8e29-414e-9e92-17affdd9160e None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:30.067428 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b6320e-8e29-414e-9e92-17affdd9160e None None] PATH_INFO: `/` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:30.068906 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 364/726] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:08:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:08:30.074515 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be24e48-25dd-47c9-b54b-10fd3df34db1 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:30.075074 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be24e48-25dd-47c9-b54b-10fd3df34db1 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:30.075565 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be24e48-25dd-47c9-b54b-10fd3df34db1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:30.098480 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-3be24e48-25dd-47c9-b54b-10fd3df34db1 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:08:30.100923 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-3be24e48-25dd-47c9-b54b-10fd3df34db1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:08:30.119190 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7b822e-1be8-4c0a-89aa-bc5fd933f131 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:30.173593 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b63461c-e384-4f3f-8e40-745aa593efd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:30.174653 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b63461c-e384-4f3f-8e40-745aa593efd0 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:30.175201 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b63461c-e384-4f3f-8e40-745aa593efd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:30.175686 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b63461c-e384-4f3f-8e40-745aa593efd0 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:30.185202 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b63461c-e384-4f3f-8e40-745aa593efd0 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:30.186928 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b63461c-e384-4f3f-8e40-745aa593efd0 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:30.198571 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-3b63461c-e384-4f3f-8e40-745aa593efd0 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 17:08:30.198571 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:30.198571 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:30.198571 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:30.198571 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:30.198571 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:30.198571 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:30.198571 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:30.198571 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:30.198571 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:30.198571 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:30.198571 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:30.198571 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:08:30.198571 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:08:30.198571 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:08:30.198571 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:08:30.200144 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:30.200144 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:30.200144 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:08:30.200144 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:08:30.200144 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:08:30.200144 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:08:30.200144 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:30.200144 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:30.200144 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:30.200144 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:30.200144 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:30.200144 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:30.200144 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:30.200144 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:30.200144 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:30.200144 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:30.200144 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:30.200144 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:30.201097 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:30.201097 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:30.201097 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:08:30.201097 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:08:30.201097 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:08:30.201097 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:08:30.201097 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:08:30.201097 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:08:30.201097 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 17:08:30.201097 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:08:30.203629 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 363/727] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Aug 24 17:08:29 2022] GET /identity/v3/users/swift => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:30.209325 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b63461c-e384-4f3f-8e40-745aa593efd0 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:30.219430 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 365/728] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:08:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 17:08:30.341380 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:30.378307 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b63461c-e384-4f3f-8e40-745aa593efd0 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:30.426394 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cd7cd8-78bf-4063-b18a-22db634aa3a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:30.427470 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cd7cd8-78bf-4063-b18a-22db634aa3a2 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:30.428146 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cd7cd8-78bf-4063-b18a-22db634aa3a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:30.428632 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cd7cd8-78bf-4063-b18a-22db634aa3a2 None admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:30.429721 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cd7cd8-78bf-4063-b18a-22db634aa3a2 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:30.430236 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cd7cd8-78bf-4063-b18a-22db634aa3a2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:30.431887 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cd7cd8-78bf-4063-b18a-22db634aa3a2 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:30.451339 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 364/729] 10.222.0.18 () {56 vars in 1135 bytes} [Wed Aug 24 17:08:30 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:30.457234 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be24e48-25dd-47c9-b54b-10fd3df34db1 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:30.509436 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173ba36f-c983-4c46-b6e5-51c0be9c089d None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:30.509767 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173ba36f-c983-4c46-b6e5-51c0be9c089d None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:30.510034 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173ba36f-c983-4c46-b6e5-51c0be9c089d None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:30.511537 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 365/730] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:08:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 17:08:30.525305 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7387a29e-6be0-4436-82f4-3b3f573e1dee None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:30.525583 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7387a29e-6be0-4436-82f4-3b3f573e1dee None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:30.525852 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7387a29e-6be0-4436-82f4-3b3f573e1dee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:30.547591 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-7387a29e-6be0-4436-82f4-3b3f573e1dee None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:08:30.550281 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-7387a29e-6be0-4436-82f4-3b3f573e1dee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:08:30.611996 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be24e48-25dd-47c9-b54b-10fd3df34db1 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:30.638814 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 366/731] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:08:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:08:30.640238 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:30.642020 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7387a29e-6be0-4436-82f4-3b3f573e1dee None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:30.671933 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8434c706-93ba-488b-8ab8-0ea5e66f97d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:30.673829 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8434c706-93ba-488b-8ab8-0ea5e66f97d7 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:30.674526 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8434c706-93ba-488b-8ab8-0ea5e66f97d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:30.675041 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8434c706-93ba-488b-8ab8-0ea5e66f97d7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:30.728702 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:30.768047 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8434c706-93ba-488b-8ab8-0ea5e66f97d7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:30.770067 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8434c706-93ba-488b-8ab8-0ea5e66f97d7 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:30.783583 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7387a29e-6be0-4436-82f4-3b3f573e1dee None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:30.830562 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2e5f5a-f07d-466b-aeed-dda96579b027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:30.833476 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2e5f5a-f07d-466b-aeed-dda96579b027 None admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:30.833608 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2e5f5a-f07d-466b-aeed-dda96579b027 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:30.835153 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2e5f5a-f07d-466b-aeed-dda96579b027 None admin] PATH_INFO: `/v3/services` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:30.836034 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2e5f5a-f07d-466b-aeed-dda96579b027 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:30.838471 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2e5f5a-f07d-466b-aeed-dda96579b027 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:30.889792 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 366/732] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 24 17:08:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 17:08:30.895238 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b8cd1c-0840-48c5-8367-c4c319beea36 None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:30.895527 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b8cd1c-0840-48c5-8367-c4c319beea36 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:30.895804 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b8cd1c-0840-48c5-8367-c4c319beea36 None None] PATH_INFO: `/` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:30.897219 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 367/733] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:08:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:08:30.897763 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b8cd1c-0840-48c5-8367-c4c319beea36 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:30.917076 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 367/734] 10.222.0.18 () {60 vars in 1141 bytes} [Wed Aug 24 17:08:30 2022] POST /identity/v3/services => generated 240 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:08:30.919291 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27818601-43f0-4f3b-b2d2-a2906b2defae None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:30.919770 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27818601-43f0-4f3b-b2d2-a2906b2defae None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:30.920040 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27818601-43f0-4f3b-b2d2-a2906b2defae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:30.933567 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:30.942174 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-27818601-43f0-4f3b-b2d2-a2906b2defae None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:08:30.943495 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-27818601-43f0-4f3b-b2d2-a2906b2defae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:08:31.034947 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 368/735] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:08:30 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:08:31.034947 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27818601-43f0-4f3b-b2d2-a2906b2defae None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:31.059780 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b8cd1c-0840-48c5-8367-c4c319beea36 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:31.085174 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:31.104738 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e347d9a-4ff5-436f-b0dd-b4bf6b4ffc00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:31.105374 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e347d9a-4ff5-436f-b0dd-b4bf6b4ffc00 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:31.106176 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e347d9a-4ff5-436f-b0dd-b4bf6b4ffc00 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:31.106458 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e347d9a-4ff5-436f-b0dd-b4bf6b4ffc00 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:31.118339 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e347d9a-4ff5-436f-b0dd-b4bf6b4ffc00 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:31.119883 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e347d9a-4ff5-436f-b0dd-b4bf6b4ffc00 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:31.135784 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-0e347d9a-4ff5-436f-b0dd-b4bf6b4ffc00 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:08:31.135784 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:31.135784 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:31.135784 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:31.135784 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:31.135784 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:31.135784 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:31.135784 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:31.135784 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:31.135784 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:31.135784 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:31.135784 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:31.135784 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:08:31.135784 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:08:31.135784 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:08:31.135784 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:08:31.136784 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:31.136784 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:31.136784 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:31.136784 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:31.136784 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:31.136784 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:31.136784 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:31.136784 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:31.136784 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:31.136784 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:31.136784 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:31.136784 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:31.136784 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:31.136784 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:31.136784 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:31.136784 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:31.136784 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:08:31.138783 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:08:31.138783 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:08:31.138783 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:08:31.138783 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 368/736] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 24 17:08:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:31.138783 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c416a6e-cc10-4f94-b211-efc75d47d146 None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:31.138783 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c416a6e-cc10-4f94-b211-efc75d47d146 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:31.138783 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c416a6e-cc10-4f94-b211-efc75d47d146 None None] PATH_INFO: `/` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:31.144914 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 369/737] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:08:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:08:31.145567 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c416a6e-cc10-4f94-b211-efc75d47d146 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:31.160762 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27818601-43f0-4f3b-b2d2-a2906b2defae None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:31.202622 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e510114c-01af-42bb-9885-b13cc4be0e02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:31.203291 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e510114c-01af-42bb-9885-b13cc4be0e02 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:31.203571 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e510114c-01af-42bb-9885-b13cc4be0e02 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:31.203848 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e510114c-01af-42bb-9885-b13cc4be0e02 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:31.216169 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e510114c-01af-42bb-9885-b13cc4be0e02 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:08:31.221172 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-e510114c-01af-42bb-9885-b13cc4be0e02 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:08:31.221172 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:31.221172 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:31.221172 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:31.221172 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:31.221172 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:31.221172 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:31.221172 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:31.221172 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:31.221172 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:31.221172 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:31.221172 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:31.221172 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:08:31.221172 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:08:31.221172 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:08:31.221172 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:08:31.222094 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:08:31.222094 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:08:31.222094 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:08:31.222094 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:08:31.222094 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:31.222094 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:31.222094 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:31.222094 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:31.222094 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:31.222094 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:31.222094 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:31.222094 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:31.222094 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:31.222094 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:31.222094 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:31.222094 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:31.223052 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:31.223052 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:31.223052 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:31.223052 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:31.223052 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:08:31.223052 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:08:31.223052 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:08:31.223052 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:08:31.223052 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:08:31.223052 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:08:31.223052 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:08:31.223052 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:08:31.223052 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 369/738] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Aug 24 17:08:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:31.224248 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b0630c-d9bf-456d-bf54-27069d6b1510 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:31.227074 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b0630c-d9bf-456d-bf54-27069d6b1510 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:31.227074 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b0630c-d9bf-456d-bf54-27069d6b1510 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:31.227427 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:31.246165 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-42b0630c-d9bf-456d-bf54-27069d6b1510 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:08:31.248042 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-42b0630c-d9bf-456d-bf54-27069d6b1510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:08:31.261105 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c416a6e-cc10-4f94-b211-efc75d47d146 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:31.297786 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd2d0d9-f85f-408b-abd0-75d12ecee7ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:31.298506 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd2d0d9-f85f-408b-abd0-75d12ecee7ab None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:31.298796 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd2d0d9-f85f-408b-abd0-75d12ecee7ab None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:31.299099 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd2d0d9-f85f-408b-abd0-75d12ecee7ab None admin] PATH_INFO: `/v3/domains` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:31.299746 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd2d0d9-f85f-408b-abd0-75d12ecee7ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:31.300049 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd2d0d9-f85f-408b-abd0-75d12ecee7ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:31.301531 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd2d0d9-f85f-408b-abd0-75d12ecee7ab None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:31.314362 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 370/739] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 24 17:08:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:08:31.319466 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd2d0d9-f85f-408b-abd0-75d12ecee7ab None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:31.331033 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 370/740] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:08:31 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 17:08:31.332298 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b0630c-d9bf-456d-bf54-27069d6b1510 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:31.369065 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:31.449862 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd2d0d9-f85f-408b-abd0-75d12ecee7ab None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:31.454036 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b0630c-d9bf-456d-bf54-27069d6b1510 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:31.489085 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:31.493237 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220f57ad-ec2c-4fdd-aa17-2d31583eb145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:31.493757 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272547d5-5164-4dad-89b8-e7ba71ff709b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:31.494363 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272547d5-5164-4dad-89b8-e7ba71ff709b None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:31.494612 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220f57ad-ec2c-4fdd-aa17-2d31583eb145 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:31.494725 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272547d5-5164-4dad-89b8-e7ba71ff709b None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:31.494919 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272547d5-5164-4dad-89b8-e7ba71ff709b None admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:31.495401 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220f57ad-ec2c-4fdd-aa17-2d31583eb145 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:31.495604 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272547d5-5164-4dad-89b8-e7ba71ff709b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:31.495907 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272547d5-5164-4dad-89b8-e7ba71ff709b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:31.496285 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220f57ad-ec2c-4fdd-aa17-2d31583eb145 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:31.497299 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272547d5-5164-4dad-89b8-e7ba71ff709b None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:31.507789 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 371/741] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 24 17:08:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:31.508767 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272547d5-5164-4dad-89b8-e7ba71ff709b None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:31.509196 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220f57ad-ec2c-4fdd-aa17-2d31583eb145 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:08:31.514395 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-220f57ad-ec2c-4fdd-aa17-2d31583eb145 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:08:31.514395 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:31.514395 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:31.514395 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:31.514395 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:31.514395 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:31.514395 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:31.514395 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:31.514395 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:31.514395 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:31.514395 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:31.514395 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:31.514395 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:08:31.514395 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:08:31.514395 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:08:31.514395 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:08:31.515370 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:08:31.515370 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:08:31.515370 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:08:31.515370 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:08:31.515370 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:31.515370 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:31.515370 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:31.515370 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:31.515370 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:31.515370 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:31.515370 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:31.515370 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:31.515370 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:31.515370 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:31.515370 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:31.515370 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:31.516222 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:31.516222 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:31.516222 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:31.516222 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:31.516222 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:08:31.516222 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:08:31.516222 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:08:31.516222 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:08:31.516222 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:08:31.516222 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:08:31.516222 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:08:31.516222 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:08:31.518640 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 371/742] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Aug 24 17:08:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:31.519762 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220f57ad-ec2c-4fdd-aa17-2d31583eb145 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:31.628376 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272547d5-5164-4dad-89b8-e7ba71ff709b None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:31.648134 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:31.656014 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220f57ad-ec2c-4fdd-aa17-2d31583eb145 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:31.669899 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c88e5e4-0266-4956-b26b-3279451673ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:31.670542 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c88e5e4-0266-4956-b26b-3279451673ab None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:31.670816 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c88e5e4-0266-4956-b26b-3279451673ab None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:31.671116 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c88e5e4-0266-4956-b26b-3279451673ab None admin] PATH_INFO: `/v3/projects/service` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:31.678063 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c88e5e4-0266-4956-b26b-3279451673ab None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:31.679584 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c88e5e4-0266-4956-b26b-3279451673ab None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:31.690892 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-6c88e5e4-0266-4956-b26b-3279451673ab None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:08:31.690892 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:31.690892 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:31.690892 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:31.690892 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:31.690892 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:31.690892 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:31.690892 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:31.690892 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:31.690892 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:31.690892 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:31.690892 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:31.690892 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:08:31.690892 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:08:31.690892 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:08:31.691819 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:08:31.691819 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:31.691819 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:31.691819 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:31.691819 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:31.691819 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:31.691819 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:31.691819 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:31.691819 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:31.691819 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:31.691819 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:31.691819 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:31.691819 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:31.691819 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:31.691819 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:31.691819 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:31.691819 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:31.692712 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:08:31.692712 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:08:31.692712 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:08:31.692712 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:08:31.692712 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:08:31.692712 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:08:31.692712 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:08:31.692712 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:08:31.692712 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 372/743] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 24 17:08:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:31.693218 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c88e5e4-0266-4956-b26b-3279451673ab None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:31.695335 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ed34f9-c7bb-44cd-895b-4f332d85a35f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:31.695964 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ed34f9-c7bb-44cd-895b-4f332d85a35f None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:31.696241 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ed34f9-c7bb-44cd-895b-4f332d85a35f None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:31.696656 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ed34f9-c7bb-44cd-895b-4f332d85a35f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:31.707800 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ed34f9-c7bb-44cd-895b-4f332d85a35f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:31.709323 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ed34f9-c7bb-44cd-895b-4f332d85a35f None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:31.725501 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-c5ed34f9-c7bb-44cd-895b-4f332d85a35f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:08:31.725501 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:31.725501 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:31.725501 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:31.725501 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:31.725501 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:31.725501 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:31.725501 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:31.725501 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:31.725501 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:31.725501 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:31.725501 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:31.725501 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:08:31.725501 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:08:31.725501 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:08:31.725501 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:08:31.726431 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:31.726431 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:31.726431 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:31.726431 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:31.726431 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:31.726431 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:31.726431 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:31.726431 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:31.726431 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:31.726431 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:31.726431 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:31.726431 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:31.726431 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:31.726431 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:31.726431 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:31.726431 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:31.726431 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:08:31.727366 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:08:31.727366 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:08:31.727366 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:08:31.727366 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 372/744] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 24 17:08:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:31.727366 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ed34f9-c7bb-44cd-895b-4f332d85a35f None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:31.755170 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:31.803426 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c88e5e4-0266-4956-b26b-3279451673ab None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:31.837117 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ed34f9-c7bb-44cd-895b-4f332d85a35f None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:31.839915 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd44aa9-02e0-4098-a690-99851370df81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:31.840526 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd44aa9-02e0-4098-a690-99851370df81 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:31.840808 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd44aa9-02e0-4098-a690-99851370df81 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:31.841082 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd44aa9-02e0-4098-a690-99851370df81 None admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:31.841740 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd44aa9-02e0-4098-a690-99851370df81 None admin] RBAC: Adding query filter params (name=service) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:31.842046 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd44aa9-02e0-4098-a690-99851370df81 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:31.843444 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd44aa9-02e0-4098-a690-99851370df81 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:31.856706 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 373/745] 10.222.0.18 () {56 vars in 1103 bytes} [Wed Aug 24 17:08:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:31.856706 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd44aa9-02e0-4098-a690-99851370df81 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:31.878221 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bac213-19d2-4b4e-8a56-0903cfa1d61a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:31.878827 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bac213-19d2-4b4e-8a56-0903cfa1d61a None admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:31.879141 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bac213-19d2-4b4e-8a56-0903cfa1d61a None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:31.879413 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bac213-19d2-4b4e-8a56-0903cfa1d61a None admin] PATH_INFO: `/v3/services` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:31.880057 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bac213-19d2-4b4e-8a56-0903cfa1d61a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:31.881430 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bac213-19d2-4b4e-8a56-0903cfa1d61a None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:31.953413 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd44aa9-02e0-4098-a690-99851370df81 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:31.957160 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 373/746] 10.222.0.18 () {60 vars in 1141 bytes} [Wed Aug 24 17:08:31 2022] POST /identity/v3/services => generated 251 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:08:31.958055 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bac213-19d2-4b4e-8a56-0903cfa1d61a None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:31.971920 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:31.977396 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:32.011229 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d5954c-6f2a-41e0-8a49-ef32ef57acf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:32.012388 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d5954c-6f2a-41e0-8a49-ef32ef57acf1 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:32.013841 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d5954c-6f2a-41e0-8a49-ef32ef57acf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:32.014317 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d5954c-6f2a-41e0-8a49-ef32ef57acf1 None admin] PATH_INFO: `/v3/projects` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:32.015434 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d5954c-6f2a-41e0-8a49-ef32ef57acf1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:32.016240 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d5954c-6f2a-41e0-8a49-ef32ef57acf1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:32.018164 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d5954c-6f2a-41e0-8a49-ef32ef57acf1 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:32.037065 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 374/747] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Aug 24 17:08:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:32.038883 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d5954c-6f2a-41e0-8a49-ef32ef57acf1 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:32.103203 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bac213-19d2-4b4e-8a56-0903cfa1d61a None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:32.142450 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947ad23d-3d14-48c6-aa55-f4580f2f6e3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:32.143118 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947ad23d-3d14-48c6-aa55-f4580f2f6e3c None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:32.143401 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947ad23d-3d14-48c6-aa55-f4580f2f6e3c None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:32.144708 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947ad23d-3d14-48c6-aa55-f4580f2f6e3c None admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:32.144708 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947ad23d-3d14-48c6-aa55-f4580f2f6e3c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:32.144708 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947ad23d-3d14-48c6-aa55-f4580f2f6e3c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:32.146036 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947ad23d-3d14-48c6-aa55-f4580f2f6e3c None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:32.161287 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 374/748] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 24 17:08:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:32.162140 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947ad23d-3d14-48c6-aa55-f4580f2f6e3c None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:32.180503 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d5954c-6f2a-41e0-8a49-ef32ef57acf1 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:32.223578 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb2e63f-8191-462c-885d-1b4f9602006a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:32.224126 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb2e63f-8191-462c-885d-1b4f9602006a None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:32.224488 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb2e63f-8191-462c-885d-1b4f9602006a None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:32.224663 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb2e63f-8191-462c-885d-1b4f9602006a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:32.238130 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb2e63f-8191-462c-885d-1b4f9602006a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:32.240054 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb2e63f-8191-462c-885d-1b4f9602006a None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:32.251107 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:32.254986 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-ddb2e63f-8191-462c-885d-1b4f9602006a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:08:32.254986 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:32.254986 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:32.254986 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:32.254986 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:32.254986 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:32.254986 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:32.254986 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:32.254986 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:32.254986 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:32.254986 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:32.254986 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:32.254986 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:08:32.254986 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:08:32.254986 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:08:32.254986 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:08:32.255920 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:32.255920 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:32.255920 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:32.255920 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:32.255920 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:32.255920 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:32.255920 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:32.255920 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:32.255920 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:32.255920 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:32.255920 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:32.255920 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:32.255920 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:32.255920 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:32.255920 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:32.255920 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:32.255920 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:08:32.257230 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:08:32.257230 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:08:32.257230 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:08:32.257230 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 375/749] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 24 17:08:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:32.257230 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb2e63f-8191-462c-885d-1b4f9602006a None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:32.289742 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:32.300867 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947ad23d-3d14-48c6-aa55-f4580f2f6e3c None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:32.349840 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9434ae7f-45ac-4b9a-a55e-9b1b750bb9e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:32.355383 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9434ae7f-45ac-4b9a-a55e-9b1b750bb9e0 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:32.355383 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9434ae7f-45ac-4b9a-a55e-9b1b750bb9e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:32.355383 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9434ae7f-45ac-4b9a-a55e-9b1b750bb9e0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:32.355383 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9434ae7f-45ac-4b9a-a55e-9b1b750bb9e0 None admin] RBAC: Adding query filter params (role.id=c5346bc54abf445f9900edf94264c850, scope.project.id=46ab158bc3644c888557781b0630b0b8, user.id=ef8caa353be34499a58cd25b72ee4032) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:32.355383 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9434ae7f-45ac-4b9a-a55e-9b1b750bb9e0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:32.356311 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9434ae7f-45ac-4b9a-a55e-9b1b750bb9e0 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:32.363581 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 375/750] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 24 17:08:32 2022] GET /identity/v3/role_assignments?user.id=ef8caa353be34499a58cd25b72ee4032&scope.project.id=46ab158bc3644c888557781b0630b0b8&role.id=c5346bc54abf445f9900edf94264c850 => generated 259 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:32.364647 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9434ae7f-45ac-4b9a-a55e-9b1b750bb9e0 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:32.377674 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb2e63f-8191-462c-885d-1b4f9602006a None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:32.481380 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f1dc42-54c4-419b-87bc-f2a2a009715e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:32.482420 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f1dc42-54c4-419b-87bc-f2a2a009715e None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:32.484299 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f1dc42-54c4-419b-87bc-f2a2a009715e None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:32.485289 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f1dc42-54c4-419b-87bc-f2a2a009715e None admin] PATH_INFO: `/v3/users/nova` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:32.497835 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f1dc42-54c4-419b-87bc-f2a2a009715e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:32.499572 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f1dc42-54c4-419b-87bc-f2a2a009715e None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:32.518814 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-f5f1dc42-54c4-419b-87bc-f2a2a009715e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 17:08:32.518814 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:32.518814 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:32.518814 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:32.518814 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:32.518814 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:32.518814 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:32.518814 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:32.518814 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:32.518814 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:32.518814 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:32.518814 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:32.518814 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:08:32.518814 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:08:32.518814 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:08:32.518814 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:08:32.519840 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:32.519840 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:32.519840 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:08:32.519840 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:08:32.519840 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:08:32.519840 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:08:32.519840 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:32.519840 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:32.519840 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:32.519840 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:32.519840 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:32.519840 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:32.519840 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:32.519840 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:32.519840 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:32.519840 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:32.519840 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:32.519840 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:32.520760 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:32.520760 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:32.520760 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:08:32.520760 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:08:32.520760 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:08:32.520760 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:08:32.520760 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:08:32.520760 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:08:32.520760 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 17:08:32.520760 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:08:32.520760 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 376/751] 10.222.0.18 () {56 vars in 1093 bytes} [Wed Aug 24 17:08:32 2022] GET /identity/v3/users/nova => generated 82 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:32.528268 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f1dc42-54c4-419b-87bc-f2a2a009715e None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:32.556658 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9434ae7f-45ac-4b9a-a55e-9b1b750bb9e0 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:32.603718 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:32.610159 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e985e7-ea9c-44d8-81ae-6b5d8f56d0e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:32.611090 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e985e7-ea9c-44d8-81ae-6b5d8f56d0e1 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:32.611636 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e985e7-ea9c-44d8-81ae-6b5d8f56d0e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:32.612124 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e985e7-ea9c-44d8-81ae-6b5d8f56d0e1 None admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:32.613080 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e985e7-ea9c-44d8-81ae-6b5d8f56d0e1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:32.613551 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e985e7-ea9c-44d8-81ae-6b5d8f56d0e1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:32.615130 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e985e7-ea9c-44d8-81ae-6b5d8f56d0e1 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:32.629974 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 376/752] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 24 17:08:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:32.635317 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e985e7-ea9c-44d8-81ae-6b5d8f56d0e1 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:32.696164 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:32.720379 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f1dc42-54c4-419b-87bc-f2a2a009715e None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:32.743644 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e985e7-ea9c-44d8-81ae-6b5d8f56d0e1 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:32.780302 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae57f08-8ffe-43da-a925-eaee55cc9123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:32.781324 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae57f08-8ffe-43da-a925-eaee55cc9123 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:32.781804 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae57f08-8ffe-43da-a925-eaee55cc9123 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:32.782272 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae57f08-8ffe-43da-a925-eaee55cc9123 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:32.789935 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae57f08-8ffe-43da-a925-eaee55cc9123 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:32.791662 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae57f08-8ffe-43da-a925-eaee55cc9123 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:32.803738 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-fae57f08-8ffe-43da-a925-eaee55cc9123 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 17:08:32.803738 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:32.803738 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:32.803738 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:32.803738 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:32.803738 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:32.803738 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:32.803738 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:32.803738 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:32.803738 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:32.803738 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:32.803738 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:32.803738 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:08:32.803738 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:08:32.803738 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:08:32.803738 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:08:32.804700 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:32.804700 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:32.804700 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:08:32.804700 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:08:32.804700 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:08:32.804700 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:08:32.804700 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:32.804700 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:32.804700 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:32.804700 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:32.804700 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:32.804700 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:32.804700 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:32.804700 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:32.804700 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:32.804700 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:32.804700 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:32.804700 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:32.805695 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:32.805695 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:32.805695 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:08:32.805695 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:08:32.805695 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:08:32.805695 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:08:32.805695 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:08:32.805695 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:08:32.805695 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 17:08:32.805695 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:08:32.808292 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 377/753] 10.222.0.18 () {56 vars in 1117 bytes} [Wed Aug 24 17:08:32 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:32.814304 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae57f08-8ffe-43da-a925-eaee55cc9123 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:32.834295 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110054f0-a56b-4844-8a33-504d66776a00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:32.835353 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110054f0-a56b-4844-8a33-504d66776a00 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:32.835885 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110054f0-a56b-4844-8a33-504d66776a00 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:32.836358 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110054f0-a56b-4844-8a33-504d66776a00 None admin] PATH_INFO: `/v3/users` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:32.839699 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110054f0-a56b-4844-8a33-504d66776a00 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:32.840295 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110054f0-a56b-4844-8a33-504d66776a00 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:32.841972 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110054f0-a56b-4844-8a33-504d66776a00 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:32.881164 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 377/754] 10.222.0.18 () {56 vars in 1133 bytes} [Wed Aug 24 17:08:32 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 354 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:32.888300 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110054f0-a56b-4844-8a33-504d66776a00 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:32.892506 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:32.945845 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae57f08-8ffe-43da-a925-eaee55cc9123 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:32.982175 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2427cba8-56c3-4837-a652-077e13011c41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:32.982928 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2427cba8-56c3-4837-a652-077e13011c41 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:32.983294 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2427cba8-56c3-4837-a652-077e13011c41 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:32.983641 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2427cba8-56c3-4837-a652-077e13011c41 None admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:32.984457 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2427cba8-56c3-4837-a652-077e13011c41 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:32.984830 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2427cba8-56c3-4837-a652-077e13011c41 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:32.986331 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2427cba8-56c3-4837-a652-077e13011c41 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:33.004933 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 378/755] 10.222.0.18 () {56 vars in 1149 bytes} [Wed Aug 24 17:08:32 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:33.006208 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82a8eca-1cdc-498c-a61d-f1b0cd0b8bac None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:33.006563 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82a8eca-1cdc-498c-a61d-f1b0cd0b8bac None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:33.006923 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82a8eca-1cdc-498c-a61d-f1b0cd0b8bac None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:33.008518 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 379/756] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:08:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:08:33.011039 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82a8eca-1cdc-498c-a61d-f1b0cd0b8bac None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:33.048760 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110054f0-a56b-4844-8a33-504d66776a00 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:33.084507 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0b44d5-e0d6-4512-88b0-92eb6ead7b49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:33.085195 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0b44d5-e0d6-4512-88b0-92eb6ead7b49 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:33.085470 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0b44d5-e0d6-4512-88b0-92eb6ead7b49 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:33.085737 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0b44d5-e0d6-4512-88b0-92eb6ead7b49 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:33.096637 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0b44d5-e0d6-4512-88b0-92eb6ead7b49 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:33.098059 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0b44d5-e0d6-4512-88b0-92eb6ead7b49 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:33.100004 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:33.114656 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-6f0b44d5-e0d6-4512-88b0-92eb6ead7b49 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:08:33.114656 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:33.114656 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:33.114656 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:33.114656 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:33.114656 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:33.114656 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:33.114656 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:33.114656 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:33.114656 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:33.114656 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:33.114656 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:33.114656 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:08:33.114656 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:08:33.114656 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:08:33.114656 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:08:33.122124 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:33.122124 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:33.122124 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:33.122124 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:33.122124 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:33.122124 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:33.122124 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:33.122124 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:33.122124 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:33.122124 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:33.122124 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:33.122124 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:33.122124 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:33.122124 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:33.122124 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:33.122124 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:33.122124 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:08:33.123143 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:08:33.123143 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:08:33.123143 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:08:33.123143 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 378/757] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 24 17:08:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:08:33.123143 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc62019-c773-47e4-b0ef-5ceac729a7dd None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:33.123143 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc62019-c773-47e4-b0ef-5ceac729a7dd None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:33.123143 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc62019-c773-47e4-b0ef-5ceac729a7dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:33.128950 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82a8eca-1cdc-498c-a61d-f1b0cd0b8bac None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:33.138136 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-1bc62019-c773-47e4-b0ef-5ceac729a7dd None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:08:33.139364 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-1bc62019-c773-47e4-b0ef-5ceac729a7dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:08:33.140432 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:33.171713 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb614bb-7341-4f84-a710-a49c53e5e380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:33.171713 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb614bb-7341-4f84-a710-a49c53e5e380 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:33.172479 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb614bb-7341-4f84-a710-a49c53e5e380 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:33.173679 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb614bb-7341-4f84-a710-a49c53e5e380 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:33.220105 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 379/758] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:08:33 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:08:33.220960 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc62019-c773-47e4-b0ef-5ceac729a7dd None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:33.250007 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb614bb-7341-4f84-a710-a49c53e5e380 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:33.251857 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb614bb-7341-4f84-a710-a49c53e5e380 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:33.385760 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc62019-c773-47e4-b0ef-5ceac729a7dd None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:33.414465 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 380/759] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 24 17:08:33 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 17:08:33.419709 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb614bb-7341-4f84-a710-a49c53e5e380 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:33.446237 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:33.463175 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:33.484695 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cdf5f7-4846-468f-aaf4-5ac9b0ce2ebc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:33.485401 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cdf5f7-4846-468f-aaf4-5ac9b0ce2ebc None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:33.485692 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cdf5f7-4846-468f-aaf4-5ac9b0ce2ebc None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:33.488438 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cdf5f7-4846-468f-aaf4-5ac9b0ce2ebc None admin] PATH_INFO: `/v3/domains` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:33.489100 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cdf5f7-4846-468f-aaf4-5ac9b0ce2ebc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:33.490029 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cdf5f7-4846-468f-aaf4-5ac9b0ce2ebc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:33.491577 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cdf5f7-4846-468f-aaf4-5ac9b0ce2ebc None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:33.504304 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 380/760] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 24 17:08:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:33.505216 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cdf5f7-4846-468f-aaf4-5ac9b0ce2ebc None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:33.568751 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb614bb-7341-4f84-a710-a49c53e5e380 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:33.613429 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3660e169-b40e-40f2-8486-773ee501dc5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:33.614062 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3660e169-b40e-40f2-8486-773ee501dc5f None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:33.614389 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3660e169-b40e-40f2-8486-773ee501dc5f None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:33.614648 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3660e169-b40e-40f2-8486-773ee501dc5f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:33.626758 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3660e169-b40e-40f2-8486-773ee501dc5f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:33.628524 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3660e169-b40e-40f2-8486-773ee501dc5f None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:33.638404 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cdf5f7-4846-468f-aaf4-5ac9b0ce2ebc None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:33.646267 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-3660e169-b40e-40f2-8486-773ee501dc5f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:08:33.646267 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:33.646267 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:33.646267 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:33.646267 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:33.646267 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:33.646267 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:33.646267 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:33.646267 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:33.646267 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:33.646267 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:33.646267 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:33.646267 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:08:33.646267 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:08:33.646267 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:08:33.646267 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:08:33.647230 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:33.647230 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:33.647230 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:33.647230 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:33.647230 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:33.647230 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:33.647230 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:33.647230 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:33.647230 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:33.647230 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:33.647230 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:33.647230 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:33.647230 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:33.647230 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:33.647230 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:33.647230 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:33.647230 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:08:33.648120 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:08:33.648120 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:08:33.648120 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:08:33.650324 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 381/761] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 24 17:08:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:33.651362 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3660e169-b40e-40f2-8486-773ee501dc5f None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:33.687119 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9c5d43-04f2-41ee-8fa9-82f223af81c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:33.688162 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9c5d43-04f2-41ee-8fa9-82f223af81c1 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:33.688639 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9c5d43-04f2-41ee-8fa9-82f223af81c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:33.689151 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9c5d43-04f2-41ee-8fa9-82f223af81c1 None admin] PATH_INFO: `/v3/services/network` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:33.689942 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9c5d43-04f2-41ee-8fa9-82f223af81c1 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:33.691950 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9c5d43-04f2-41ee-8fa9-82f223af81c1 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:33.708326 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-ed9c5d43-04f2-41ee-8fa9-82f223af81c1 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 17:08:33.708326 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:33.708326 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 17:08:33.708326 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 17:08:33.708326 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 17:08:33.708326 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 17:08:33.708326 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 17:08:33.708326 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:08:33.708326 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 17:08:33.708326 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:08:33.708326 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 17:08:33.708326 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:08:33.708326 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:33.708326 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:33.708326 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:33.708326 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:33.708326 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:33.709806 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:33.709806 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:33.709806 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:33.709806 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:33.709806 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:33.709806 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:33.709806 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 17:08:33.709806 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 17:08:33.709806 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 17:08:33.709806 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 17:08:33.709806 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:33.709806 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:33.709806 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:33.709806 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:33.709806 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:33.709806 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:33.710646 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:33.710646 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:33.710646 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:33.710646 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:33.710646 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:33.710646 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:33.710646 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:33.710646 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:33.710646 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:33.710646 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:33.710646 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 17:08:33.710646 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:08:33.710646 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 17:08:33.710646 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:08:33.718175 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 381/762] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 24 17:08:33 2022] GET /identity/v3/services/network => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:33.719359 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9c5d43-04f2-41ee-8fa9-82f223af81c1 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:33.719816 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:33.773456 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:33.775000 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3660e169-b40e-40f2-8486-773ee501dc5f None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:33.818919 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bbda1c-61f5-49e0-aec3-25e61ca9dd6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:33.819556 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bbda1c-61f5-49e0-aec3-25e61ca9dd6d None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:33.819835 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bbda1c-61f5-49e0-aec3-25e61ca9dd6d None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:33.820236 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bbda1c-61f5-49e0-aec3-25e61ca9dd6d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:33.828681 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9c5d43-04f2-41ee-8fa9-82f223af81c1 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:33.829239 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bbda1c-61f5-49e0-aec3-25e61ca9dd6d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:33.830969 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bbda1c-61f5-49e0-aec3-25e61ca9dd6d None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:33.847188 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-09bbda1c-61f5-49e0-aec3-25e61ca9dd6d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:08:33.847188 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:33.847188 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:33.847188 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:33.847188 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:33.847188 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:33.847188 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:33.847188 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:33.847188 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:33.847188 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:33.847188 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:33.847188 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:33.847188 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:08:33.847188 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:08:33.847188 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:08:33.848127 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:08:33.848127 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:33.848127 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:33.848127 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:33.848127 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:33.848127 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:33.848127 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:33.848127 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:33.848127 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:33.848127 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:33.848127 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:33.848127 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:33.848127 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:33.848127 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:33.848127 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:33.848127 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:33.848127 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:33.849029 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:08:33.849029 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:08:33.849029 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:08:33.849029 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:08:33.849029 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:08:33.849029 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:08:33.849029 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:08:33.849029 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:08:33.852128 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 382/763] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 24 17:08:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:33.853133 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bbda1c-61f5-49e0-aec3-25e61ca9dd6d None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:33.878466 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669c48b6-d16e-4a73-bef0-624a852c82fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:33.879108 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669c48b6-d16e-4a73-bef0-624a852c82fe None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:33.879382 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669c48b6-d16e-4a73-bef0-624a852c82fe None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:33.879662 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669c48b6-d16e-4a73-bef0-624a852c82fe None admin] PATH_INFO: `/v3/domains` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:33.880563 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669c48b6-d16e-4a73-bef0-624a852c82fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:33.880874 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669c48b6-d16e-4a73-bef0-624a852c82fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:33.882482 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669c48b6-d16e-4a73-bef0-624a852c82fe None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:33.899114 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 382/764] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 24 17:08:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:33.900204 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669c48b6-d16e-4a73-bef0-624a852c82fe None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:33.973908 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bbda1c-61f5-49e0-aec3-25e61ca9dd6d None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:33.997322 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:34.013543 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7725d0aa-9f86-48ff-8c59-9f8c3acc9bb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:34.014555 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7725d0aa-9f86-48ff-8c59-9f8c3acc9bb2 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:34.015064 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7725d0aa-9f86-48ff-8c59-9f8c3acc9bb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:34.015535 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7725d0aa-9f86-48ff-8c59-9f8c3acc9bb2 None admin] PATH_INFO: `/v3/services` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:34.016681 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7725d0aa-9f86-48ff-8c59-9f8c3acc9bb2 None admin] RBAC: Adding query filter params (name=network) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:34.017459 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7725d0aa-9f86-48ff-8c59-9f8c3acc9bb2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:34.019309 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7725d0aa-9f86-48ff-8c59-9f8c3acc9bb2 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:34.030051 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 383/765] 10.222.0.18 () {56 vars in 1112 bytes} [Wed Aug 24 17:08:33 2022] GET /identity/v3/services?name=network => generated 124 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:34.031345 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7725d0aa-9f86-48ff-8c59-9f8c3acc9bb2 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:34.033895 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669c48b6-d16e-4a73-bef0-624a852c82fe None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:34.046447 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:34.077403 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598daaa5-cf51-454b-b0c5-23d8e370bb5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:34.078011 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598daaa5-cf51-454b-b0c5-23d8e370bb5a None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:34.078286 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598daaa5-cf51-454b-b0c5-23d8e370bb5a None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:34.078556 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598daaa5-cf51-454b-b0c5-23d8e370bb5a None admin] PATH_INFO: `/v3/projects` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:34.079605 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598daaa5-cf51-454b-b0c5-23d8e370bb5a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:34.080299 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598daaa5-cf51-454b-b0c5-23d8e370bb5a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:34.083386 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598daaa5-cf51-454b-b0c5-23d8e370bb5a None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:34.099726 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 383/766] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Aug 24 17:08:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:34.100895 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598daaa5-cf51-454b-b0c5-23d8e370bb5a None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:34.171654 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7725d0aa-9f86-48ff-8c59-9f8c3acc9bb2 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:34.207120 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c647224a-c4a3-468f-aac1-861b3d62f6e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:34.207766 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c647224a-c4a3-468f-aac1-861b3d62f6e6 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:34.208037 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c647224a-c4a3-468f-aac1-861b3d62f6e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:34.208309 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c647224a-c4a3-468f-aac1-861b3d62f6e6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:34.209039 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598daaa5-cf51-454b-b0c5-23d8e370bb5a None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:34.217561 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c647224a-c4a3-468f-aac1-861b3d62f6e6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:34.219036 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c647224a-c4a3-468f-aac1-861b3d62f6e6 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:34.230776 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-c647224a-c4a3-468f-aac1-861b3d62f6e6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:08:34.230776 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:34.230776 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:34.230776 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:34.230776 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:34.230776 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:34.230776 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:34.230776 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:34.230776 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:34.230776 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:34.230776 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:34.230776 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:34.230776 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:08:34.230776 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:08:34.230776 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:08:34.231971 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:08:34.231971 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:34.231971 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:34.231971 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:34.231971 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:34.231971 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:34.231971 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:34.231971 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:34.231971 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:34.231971 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:34.231971 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:34.231971 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:34.231971 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:34.231971 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:34.231971 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:34.231971 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:34.231971 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:34.232967 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:08:34.232967 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:08:34.232967 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:08:34.232967 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:08:34.232967 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:08:34.232967 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:08:34.232967 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:08:34.232967 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:08:34.236556 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 384/767] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 24 17:08:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:34.237366 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c647224a-c4a3-468f-aac1-861b3d62f6e6 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:34.254207 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d36f43a-bdc7-4305-932d-8c07a6ea691e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:34.254826 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d36f43a-bdc7-4305-932d-8c07a6ea691e None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:34.255112 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d36f43a-bdc7-4305-932d-8c07a6ea691e None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:34.255384 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d36f43a-bdc7-4305-932d-8c07a6ea691e None admin] PATH_INFO: `/v3/services` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:34.256011 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d36f43a-bdc7-4305-932d-8c07a6ea691e None admin] RBAC: Adding query filter params (type=network) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:34.256309 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d36f43a-bdc7-4305-932d-8c07a6ea691e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:34.257723 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d36f43a-bdc7-4305-932d-8c07a6ea691e None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:34.268162 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 384/768] 10.222.0.18 () {56 vars in 1112 bytes} [Wed Aug 24 17:08:34 2022] GET /identity/v3/services?type=network => generated 350 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:34.268162 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e170044-a47c-4681-a383-2ae4ccaabfed None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:34.268162 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e170044-a47c-4681-a383-2ae4ccaabfed None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:34.268162 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e170044-a47c-4681-a383-2ae4ccaabfed None None] PATH_INFO: `/` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:34.268162 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 385/769] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:08:34 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:08:34.268162 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e170044-a47c-4681-a383-2ae4ccaabfed None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:34.294948 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:34.299389 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:34.358415 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c647224a-c4a3-468f-aac1-861b3d62f6e6 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:34.373515 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e170044-a47c-4681-a383-2ae4ccaabfed None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:34.397245 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0571dea3-57b1-4cfb-b654-584e97528c2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:34.397920 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0571dea3-57b1-4cfb-b654-584e97528c2e None admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:34.398199 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0571dea3-57b1-4cfb-b654-584e97528c2e None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:34.398473 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0571dea3-57b1-4cfb-b654-584e97528c2e None admin] PATH_INFO: `/v3/projects/46ab158bc3644c888557781b0630b0b8/users/557aca9255cb4bbb8af52e796c5c5177/roles/c5346bc54abf445f9900edf94264c850` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:34.400425 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0571dea3-57b1-4cfb-b654-584e97528c2e None admin] RBAC: Authorizing `identity:create_grant(project_id=46ab158bc3644c888557781b0630b0b8, user_id=557aca9255cb4bbb8af52e796c5c5177, role_id=c5346bc54abf445f9900edf94264c850)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:34.401868 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0571dea3-57b1-4cfb-b654-584e97528c2e None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:34.411429 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d23ff32-7bc7-4a55-891c-a2a537d12e6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:34.412023 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d23ff32-7bc7-4a55-891c-a2a537d12e6d None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:34.412299 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d23ff32-7bc7-4a55-891c-a2a537d12e6d None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:34.412573 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d23ff32-7bc7-4a55-891c-a2a537d12e6d None admin] PATH_INFO: `/v3/projects` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:34.420397 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d23ff32-7bc7-4a55-891c-a2a537d12e6d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:34.420397 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d23ff32-7bc7-4a55-891c-a2a537d12e6d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:34.420397 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d23ff32-7bc7-4a55-891c-a2a537d12e6d None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:34.428038 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 386/770] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Aug 24 17:08:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:34.428774 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d23ff32-7bc7-4a55-891c-a2a537d12e6d None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:34.480054 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 385/771] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Aug 24 17:08:34 2022] PUT /identity/v3/projects/46ab158bc3644c888557781b0630b0b8/users/557aca9255cb4bbb8af52e796c5c5177/roles/c5346bc54abf445f9900edf94264c850 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:08:34.481724 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e7ce63-d91c-4dc5-ad52-b357b7ac551f None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:34.482012 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e7ce63-d91c-4dc5-ad52-b357b7ac551f None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:34.482289 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e7ce63-d91c-4dc5-ad52-b357b7ac551f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:34.502132 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-45e7ce63-d91c-4dc5-ad52-b357b7ac551f None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:08:34.503420 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-45e7ce63-d91c-4dc5-ad52-b357b7ac551f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:08:34.546302 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:34.556023 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d23ff32-7bc7-4a55-891c-a2a537d12e6d None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:34.601089 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e78a84-d77b-44ce-8df8-ad5524166726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:34.601762 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e78a84-d77b-44ce-8df8-ad5524166726 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:34.602043 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e78a84-d77b-44ce-8df8-ad5524166726 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:34.602621 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e78a84-d77b-44ce-8df8-ad5524166726 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:34.602730 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 386/772] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:08:34 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:08:34.603488 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e78a84-d77b-44ce-8df8-ad5524166726 None admin] RBAC: Adding query filter params (interface=public, service_id=2bead5829f1a42138906b656e58f2ea9, region_id=RegionOne) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:34.603810 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e78a84-d77b-44ce-8df8-ad5524166726 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:34.605261 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e78a84-d77b-44ce-8df8-ad5524166726 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:34.612312 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e7ce63-d91c-4dc5-ad52-b357b7ac551f None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:34.619340 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 387/773] 10.222.0.18 () {56 vars in 1251 bytes} [Wed Aug 24 17:08:34 2022] GET /identity/v3/endpoints?service_id=2bead5829f1a42138906b656e58f2ea9&interface=public®ion_id=RegionOne => generated 194 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:34.619340 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2062fd-a53b-4809-8f2f-47e4795d3c21 None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:34.619340 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2062fd-a53b-4809-8f2f-47e4795d3c21 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:34.619340 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2062fd-a53b-4809-8f2f-47e4795d3c21 None None] PATH_INFO: `/` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:34.623458 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 388/774] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:08:34 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:08:34.624909 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a771161-079d-4b76-9760-c1bb3d41fed3 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:34.625204 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a771161-079d-4b76-9760-c1bb3d41fed3 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:34.625485 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a771161-079d-4b76-9760-c1bb3d41fed3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:34.649825 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-7a771161-079d-4b76-9760-c1bb3d41fed3 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:08:34.652914 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-7a771161-079d-4b76-9760-c1bb3d41fed3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:08:34.664559 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:34.753841 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 389/775] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:08:34 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:08:34.755578 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a771161-079d-4b76-9760-c1bb3d41fed3 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:34.765788 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e7ce63-d91c-4dc5-ad52-b357b7ac551f None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:34.839917 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdba5d03-22c2-48e5-bb13-1639ef142031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:34.840991 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdba5d03-22c2-48e5-bb13-1639ef142031 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:34.842680 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdba5d03-22c2-48e5-bb13-1639ef142031 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:34.843449 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdba5d03-22c2-48e5-bb13-1639ef142031 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:34.845100 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdba5d03-22c2-48e5-bb13-1639ef142031 None admin] RBAC: Adding query filter params (role.id=e7db2a847f56438bac13d0ff21c96b73, scope.project.id=46ab158bc3644c888557781b0630b0b8, user.id=0ccc6723f7514e52863d102de3839951) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:34.845802 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdba5d03-22c2-48e5-bb13-1639ef142031 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:34.848994 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdba5d03-22c2-48e5-bb13-1639ef142031 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:34.864728 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 387/776] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 24 17:08:34 2022] GET /identity/v3/role_assignments?user.id=0ccc6723f7514e52863d102de3839951&scope.project.id=46ab158bc3644c888557781b0630b0b8&role.id=e7db2a847f56438bac13d0ff21c96b73 => generated 259 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:34.897494 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdba5d03-22c2-48e5-bb13-1639ef142031 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:34.932837 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:34.960267 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a771161-079d-4b76-9760-c1bb3d41fed3 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:35.033688 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4063496-4244-4b09-bb47-a49c1aa51012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:35.034238 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4063496-4244-4b09-bb47-a49c1aa51012 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:35.034737 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4063496-4244-4b09-bb47-a49c1aa51012 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:35.035043 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4063496-4244-4b09-bb47-a49c1aa51012 None admin] PATH_INFO: `/v3/services` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:35.035664 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4063496-4244-4b09-bb47-a49c1aa51012 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:35.037063 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4063496-4244-4b09-bb47-a49c1aa51012 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:35.044387 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 390/777] 10.222.0.18 () {56 vars in 1087 bytes} [Wed Aug 24 17:08:34 2022] GET /identity/v3/services => generated 772 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:35.088406 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdba5d03-22c2-48e5-bb13-1639ef142031 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:35.091326 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:35.134997 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5db108f-59c8-47a6-aad9-b4d45419e276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:35.135648 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5db108f-59c8-47a6-aad9-b4d45419e276 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:35.135927 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5db108f-59c8-47a6-aad9-b4d45419e276 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:35.136198 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5db108f-59c8-47a6-aad9-b4d45419e276 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:35.137396 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5db108f-59c8-47a6-aad9-b4d45419e276 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:35.140794 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5db108f-59c8-47a6-aad9-b4d45419e276 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:35.147696 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4063496-4244-4b09-bb47-a49c1aa51012 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:35.158074 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-c5db108f-59c8-47a6-aad9-b4d45419e276 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 17:08:35.158074 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:35.158074 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 17:08:35.158074 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 17:08:35.158074 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 17:08:35.158074 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 17:08:35.158074 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 17:08:35.158074 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:08:35.158074 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 17:08:35.158074 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:08:35.158074 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 17:08:35.158074 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:08:35.158074 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:35.158074 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:35.158074 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:35.158074 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:35.158074 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:35.159145 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:35.159145 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:35.159145 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:35.159145 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:35.159145 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:35.159145 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:35.159145 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 17:08:35.159145 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 17:08:35.159145 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 17:08:35.159145 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 17:08:35.159145 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:35.159145 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:35.159145 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:35.159145 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:35.159145 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:35.159145 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:35.159969 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:35.159969 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:35.159969 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:35.159969 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:35.159969 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:35.159969 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:35.159969 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:35.159969 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:35.159969 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:35.159969 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:35.159969 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 17:08:35.159969 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:08:35.159969 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 17:08:35.159969 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:08:35.159969 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 388/778] 10.222.0.18 () {56 vars in 1129 bytes} [Wed Aug 24 17:08:34 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:08:35.165614 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5db108f-59c8-47a6-aad9-b4d45419e276 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:35.341995 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5db108f-59c8-47a6-aad9-b4d45419e276 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:35.348549 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4063496-4244-4b09-bb47-a49c1aa51012 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:35.398042 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ba62af-d93c-48aa-8789-1d444dd77042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:35.398662 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ba62af-d93c-48aa-8789-1d444dd77042 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:35.399192 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ba62af-d93c-48aa-8789-1d444dd77042 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:35.399795 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ba62af-d93c-48aa-8789-1d444dd77042 None admin] PATH_INFO: `/v3/services` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:35.400989 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ba62af-d93c-48aa-8789-1d444dd77042 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:35.401348 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ba62af-d93c-48aa-8789-1d444dd77042 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:35.403407 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ba62af-d93c-48aa-8789-1d444dd77042 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:35.404278 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f07aed-4ec0-40cd-bb90-68e76c20cb50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:35.404929 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f07aed-4ec0-40cd-bb90-68e76c20cb50 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:35.405211 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f07aed-4ec0-40cd-bb90-68e76c20cb50 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:35.405481 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f07aed-4ec0-40cd-bb90-68e76c20cb50 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:35.409736 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 389/779] 10.222.0.18 () {56 vars in 1124 bytes} [Wed Aug 24 17:08:35 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:08:35.415428 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ba62af-d93c-48aa-8789-1d444dd77042 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:35.421226 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f07aed-4ec0-40cd-bb90-68e76c20cb50 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:08:35.426224 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-a6f07aed-4ec0-40cd-bb90-68e76c20cb50 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:08:35.426224 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:35.426224 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:35.426224 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:35.426224 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:35.426224 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:35.426224 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:35.426224 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:35.426224 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:35.426224 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:35.426224 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:35.426224 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:35.426224 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:08:35.426224 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:08:35.426224 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:08:35.426224 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:08:35.427312 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:08:35.427312 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:08:35.427312 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:08:35.427312 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:08:35.427312 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:35.427312 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:35.427312 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:35.427312 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:35.427312 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:35.427312 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:35.427312 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:35.427312 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:35.427312 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:35.427312 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:35.427312 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:35.427312 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:35.428510 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:35.428510 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:35.428510 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:35.428510 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:35.428510 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:08:35.428510 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:08:35.428510 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:08:35.428510 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:08:35.428510 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:08:35.428510 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:08:35.428510 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:08:35.428510 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:08:35.428510 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 391/780] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Aug 24 17:08:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:35.444065 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f07aed-4ec0-40cd-bb90-68e76c20cb50 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:35.451797 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:35.472784 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:35.531425 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ba62af-d93c-48aa-8789-1d444dd77042 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:35.572377 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015831ef-a7d0-4857-867a-7ff33520612b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:35.576036 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015831ef-a7d0-4857-867a-7ff33520612b None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:35.576575 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015831ef-a7d0-4857-867a-7ff33520612b None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:35.577042 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015831ef-a7d0-4857-867a-7ff33520612b None admin] PATH_INFO: `/v3/services` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:35.577866 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015831ef-a7d0-4857-867a-7ff33520612b None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:35.578380 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015831ef-a7d0-4857-867a-7ff33520612b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:35.580167 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015831ef-a7d0-4857-867a-7ff33520612b None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:35.590859 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 390/781] 10.222.0.18 () {56 vars in 1124 bytes} [Wed Aug 24 17:08:35 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:08:35.598061 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015831ef-a7d0-4857-867a-7ff33520612b None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:35.681716 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f07aed-4ec0-40cd-bb90-68e76c20cb50 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:35.731362 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:35.750288 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015831ef-a7d0-4857-867a-7ff33520612b None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:35.772831 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93721350-40c7-4dec-92d9-4d7ecbb1c4ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:35.773444 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93721350-40c7-4dec-92d9-4d7ecbb1c4ad None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:35.773725 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93721350-40c7-4dec-92d9-4d7ecbb1c4ad None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:35.774000 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93721350-40c7-4dec-92d9-4d7ecbb1c4ad None admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:35.774688 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93721350-40c7-4dec-92d9-4d7ecbb1c4ad None admin] RBAC: Adding query filter params (name=admin) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:35.775012 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93721350-40c7-4dec-92d9-4d7ecbb1c4ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:35.776381 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93721350-40c7-4dec-92d9-4d7ecbb1c4ad None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:35.796216 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 392/782] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 24 17:08:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:35.802083 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93721350-40c7-4dec-92d9-4d7ecbb1c4ad None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:35.826685 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb35d0e-293a-49a1-a835-12a04d83d50d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:35.827331 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb35d0e-293a-49a1-a835-12a04d83d50d None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:35.827614 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb35d0e-293a-49a1-a835-12a04d83d50d None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:35.827889 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb35d0e-293a-49a1-a835-12a04d83d50d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:35.828592 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb35d0e-293a-49a1-a835-12a04d83d50d None admin] RBAC: Adding query filter params (interface=public, service_id=2010ac9e32db4795941efc7b5c042bdc, region_id=RegionOne) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:35.828899 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb35d0e-293a-49a1-a835-12a04d83d50d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:35.830262 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb35d0e-293a-49a1-a835-12a04d83d50d None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:35.852352 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 391/783] 10.222.0.18 () {56 vars in 1251 bytes} [Wed Aug 24 17:08:35 2022] GET /identity/v3/endpoints?service_id=2010ac9e32db4795941efc7b5c042bdc&interface=public®ion_id=RegionOne => generated 194 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:35.858386 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb35d0e-293a-49a1-a835-12a04d83d50d None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:35.931861 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:35.967448 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93721350-40c7-4dec-92d9-4d7ecbb1c4ad None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:36.002890 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b70ccd7-8b13-4158-962d-bef02fcf09ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:36.003558 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b70ccd7-8b13-4158-962d-bef02fcf09ab None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:36.003838 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b70ccd7-8b13-4158-962d-bef02fcf09ab None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:36.004115 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b70ccd7-8b13-4158-962d-bef02fcf09ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:36.014919 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b70ccd7-8b13-4158-962d-bef02fcf09ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:36.016390 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b70ccd7-8b13-4158-962d-bef02fcf09ab None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:36.031315 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-9b70ccd7-8b13-4158-962d-bef02fcf09ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:08:36.031315 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:36.031315 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:36.031315 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:36.031315 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:36.031315 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:36.031315 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:36.031315 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:36.031315 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:36.031315 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:36.031315 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:36.031315 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:36.031315 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:08:36.031315 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:08:36.031315 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:08:36.031315 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:08:36.032275 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:36.032275 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:36.032275 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:36.032275 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:36.032275 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:36.032275 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:36.032275 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:36.032275 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:36.032275 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:36.032275 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:36.032275 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:36.032275 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:36.032275 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:36.032275 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:36.032275 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:36.032275 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:36.032275 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:08:36.033145 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:08:36.033145 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:08:36.033145 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:08:36.033145 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 393/784] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 24 17:08:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:36.038768 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b70ccd7-8b13-4158-962d-bef02fcf09ab None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:36.108908 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb35d0e-293a-49a1-a835-12a04d83d50d None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:36.156670 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b70ccd7-8b13-4158-962d-bef02fcf09ab None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:36.187370 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:36.195503 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aac3327-6958-42ae-b93e-b2eb3a7baab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:36.196396 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aac3327-6958-42ae-b93e-b2eb3a7baab6 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:36.196861 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aac3327-6958-42ae-b93e-b2eb3a7baab6 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:36.197483 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aac3327-6958-42ae-b93e-b2eb3a7baab6 None admin] PATH_INFO: `/v3/domains` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:36.198328 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aac3327-6958-42ae-b93e-b2eb3a7baab6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:36.198826 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aac3327-6958-42ae-b93e-b2eb3a7baab6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:36.200941 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aac3327-6958-42ae-b93e-b2eb3a7baab6 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:36.220000 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 394/785] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 24 17:08:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:36.226230 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d127f463-f5f6-4e81-a501-51a754432bbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:36.226863 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d127f463-f5f6-4e81-a501-51a754432bbd None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:36.227168 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d127f463-f5f6-4e81-a501-51a754432bbd None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:36.227435 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d127f463-f5f6-4e81-a501-51a754432bbd None admin] PATH_INFO: `/v3/services` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:36.228053 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d127f463-f5f6-4e81-a501-51a754432bbd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:36.229445 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d127f463-f5f6-4e81-a501-51a754432bbd None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:36.233757 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aac3327-6958-42ae-b93e-b2eb3a7baab6 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:36.247842 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 392/786] 10.222.0.18 () {56 vars in 1087 bytes} [Wed Aug 24 17:08:35 2022] GET /identity/v3/services => generated 772 bytes in 390 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:08:36.394000 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aac3327-6958-42ae-b93e-b2eb3a7baab6 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:36.435227 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5dc8f4-735e-4e7f-8473-5fc04b06eb83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:36.435857 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5dc8f4-735e-4e7f-8473-5fc04b06eb83 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:36.436136 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5dc8f4-735e-4e7f-8473-5fc04b06eb83 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:36.436410 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5dc8f4-735e-4e7f-8473-5fc04b06eb83 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:36.443622 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5dc8f4-735e-4e7f-8473-5fc04b06eb83 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:36.445161 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5dc8f4-735e-4e7f-8473-5fc04b06eb83 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:36.457756 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-4f5dc8f4-735e-4e7f-8473-5fc04b06eb83 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 17:08:36.457756 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:36.457756 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:36.457756 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:36.457756 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:36.457756 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:36.457756 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:36.457756 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:36.457756 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:36.457756 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:36.457756 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:36.457756 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:36.457756 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:08:36.457756 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:08:36.457756 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:08:36.457756 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:08:36.458743 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:36.458743 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:36.458743 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:08:36.458743 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:08:36.458743 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:08:36.458743 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:08:36.458743 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:36.458743 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:36.458743 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:36.458743 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:36.458743 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:36.458743 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:36.458743 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:36.458743 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:36.458743 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:36.458743 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:36.458743 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:36.458743 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:36.459740 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:36.459740 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:36.459740 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:08:36.459740 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:08:36.459740 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:08:36.459740 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:08:36.459740 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:08:36.459740 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:08:36.459740 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 17:08:36.459740 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:08:36.459740 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 395/787] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Aug 24 17:08:36 2022] GET /identity/v3/users/swift => generated 83 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:08:36.464568 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d127f463-f5f6-4e81-a501-51a754432bbd None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:36.492229 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:36.583007 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d127f463-f5f6-4e81-a501-51a754432bbd None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:36.626769 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7135baf1-f3bc-4ecf-8b59-dee9bc7f563a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:36.627933 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7135baf1-f3bc-4ecf-8b59-dee9bc7f563a None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:36.628390 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7135baf1-f3bc-4ecf-8b59-dee9bc7f563a None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:36.628835 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7135baf1-f3bc-4ecf-8b59-dee9bc7f563a None admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:36.629744 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7135baf1-f3bc-4ecf-8b59-dee9bc7f563a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:36.630250 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7135baf1-f3bc-4ecf-8b59-dee9bc7f563a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:36.631869 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7135baf1-f3bc-4ecf-8b59-dee9bc7f563a None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:36.659132 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 393/788] 10.222.0.18 () {56 vars in 1135 bytes} [Wed Aug 24 17:08:36 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:36.664945 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5dc8f4-735e-4e7f-8473-5fc04b06eb83 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:36.734252 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:36.807061 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5dc8f4-735e-4e7f-8473-5fc04b06eb83 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:36.842987 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880e99d9-4355-4c42-a75a-f8c78611fc9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:36.843627 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880e99d9-4355-4c42-a75a-f8c78611fc9c None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:36.843965 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880e99d9-4355-4c42-a75a-f8c78611fc9c None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:36.844245 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880e99d9-4355-4c42-a75a-f8c78611fc9c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:36.862583 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880e99d9-4355-4c42-a75a-f8c78611fc9c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:36.864092 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880e99d9-4355-4c42-a75a-f8c78611fc9c None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:36.883183 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-880e99d9-4355-4c42-a75a-f8c78611fc9c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:08:36.883183 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:36.883183 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:36.883183 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:36.883183 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:36.883183 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:36.883183 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:36.883183 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:36.883183 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:36.883183 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:36.883183 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:36.883183 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:36.883183 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:08:36.883183 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:08:36.883183 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:08:36.883183 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:08:36.884245 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:36.884245 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:36.884245 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:36.884245 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:36.884245 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:36.884245 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:36.884245 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:36.884245 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:36.884245 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:36.884245 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:36.884245 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:36.884245 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:36.884245 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:36.884245 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:36.884245 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:36.884245 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:36.884245 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:08:36.885123 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:08:36.885123 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:08:36.885123 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:08:36.885123 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 396/789] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 24 17:08:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:36.892074 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7135baf1-f3bc-4ecf-8b59-dee9bc7f563a None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:36.923814 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ae81f5-69e1-44f0-8899-37738f7614b4 None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:36.924113 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ae81f5-69e1-44f0-8899-37738f7614b4 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:36.924279 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ae81f5-69e1-44f0-8899-37738f7614b4 None None] PATH_INFO: `/` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:36.926148 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 397/790] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:08:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:08:36.935443 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf719034-8542-4b77-bf5e-db1b25e2de71 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:36.935734 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf719034-8542-4b77-bf5e-db1b25e2de71 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:36.936018 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf719034-8542-4b77-bf5e-db1b25e2de71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:36.976990 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-bf719034-8542-4b77-bf5e-db1b25e2de71 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:08:36.979593 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-bf719034-8542-4b77-bf5e-db1b25e2de71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:08:37.023261 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:37.070943 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7135baf1-f3bc-4ecf-8b59-dee9bc7f563a None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:37.107099 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 398/791] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:08:36 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:08:37.125758 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c6acc7-e76d-43c5-a130-3b7ad08983d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:37.126631 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c6acc7-e76d-43c5-a130-3b7ad08983d8 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:37.126922 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c6acc7-e76d-43c5-a130-3b7ad08983d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:37.127198 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c6acc7-e76d-43c5-a130-3b7ad08983d8 None admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:37.127843 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c6acc7-e76d-43c5-a130-3b7ad08983d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:37.128141 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c6acc7-e76d-43c5-a130-3b7ad08983d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:37.129493 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c6acc7-e76d-43c5-a130-3b7ad08983d8 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:37.148529 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 394/792] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 24 17:08:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:37.154664 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf719034-8542-4b77-bf5e-db1b25e2de71 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:37.187207 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29810b9f-0bb2-40a7-abfc-0f06fa3c7413 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:37.187815 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29810b9f-0bb2-40a7-abfc-0f06fa3c7413 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:37.188291 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29810b9f-0bb2-40a7-abfc-0f06fa3c7413 None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:37.189394 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:37.195004 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 395/793] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:08:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:08:37.208426 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb13bed-ff99-43fc-98fb-ab32ae473dfe None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:37.208759 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb13bed-ff99-43fc-98fb-ab32ae473dfe None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:37.209035 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb13bed-ff99-43fc-98fb-ab32ae473dfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:37.239293 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-cfb13bed-ff99-43fc-98fb-ab32ae473dfe None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:08:37.240477 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-cfb13bed-ff99-43fc-98fb-ab32ae473dfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:08:37.268584 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf719034-8542-4b77-bf5e-db1b25e2de71 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:37.302853 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaab8de5-cf1a-41b6-be3a-b7f654753edf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:37.303846 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaab8de5-cf1a-41b6-be3a-b7f654753edf None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:37.304322 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaab8de5-cf1a-41b6-be3a-b7f654753edf None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:37.304812 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaab8de5-cf1a-41b6-be3a-b7f654753edf None admin] PATH_INFO: `/v3/projects/service` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:37.312343 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaab8de5-cf1a-41b6-be3a-b7f654753edf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:37.314029 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaab8de5-cf1a-41b6-be3a-b7f654753edf None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:37.325931 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-aaab8de5-cf1a-41b6-be3a-b7f654753edf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:08:37.325931 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:37.325931 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:37.325931 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:37.325931 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:37.325931 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:37.325931 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:37.325931 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:37.325931 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:37.325931 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:37.325931 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:37.325931 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:37.325931 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:08:37.325931 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:08:37.325931 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:08:37.326818 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:08:37.326818 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:37.326818 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:37.326818 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:37.326818 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:37.326818 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:37.326818 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:37.326818 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:37.326818 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:37.326818 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:37.326818 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:37.326818 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:37.326818 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:37.326818 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:37.326818 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:37.326818 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:37.326818 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:37.327877 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:08:37.327877 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:08:37.327877 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:08:37.327877 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:08:37.327877 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:08:37.327877 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:08:37.327877 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:08:37.327877 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:08:37.329834 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 399/794] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 24 17:08:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:08:37.335412 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaab8de5-cf1a-41b6-be3a-b7f654753edf None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:37.406508 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 396/795] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:08:37 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:08:37.427267 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:37.431692 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484e4a84-df19-4ad5-b91b-188b7738a9ec None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:37.431982 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484e4a84-df19-4ad5-b91b-188b7738a9ec None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:37.432259 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484e4a84-df19-4ad5-b91b-188b7738a9ec None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:37.437035 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaab8de5-cf1a-41b6-be3a-b7f654753edf None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:37.437655 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 397/796] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:08:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:08:37.444201 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbccf172-48b8-4da6-85be-5a9e72592974 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:37.444486 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbccf172-48b8-4da6-85be-5a9e72592974 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:37.444762 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbccf172-48b8-4da6-85be-5a9e72592974 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:37.471994 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-dbccf172-48b8-4da6-85be-5a9e72592974 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:08:37.474190 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-dbccf172-48b8-4da6-85be-5a9e72592974 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:08:37.480790 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbe645d-d9cf-4a9d-9a19-8e89e5b5b1a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:37.481576 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbe645d-d9cf-4a9d-9a19-8e89e5b5b1a3 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:37.481994 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbe645d-d9cf-4a9d-9a19-8e89e5b5b1a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:37.482425 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbe645d-d9cf-4a9d-9a19-8e89e5b5b1a3 None admin] PATH_INFO: `/v3/projects` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:37.483281 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbe645d-d9cf-4a9d-9a19-8e89e5b5b1a3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:37.483726 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbe645d-d9cf-4a9d-9a19-8e89e5b5b1a3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:37.485284 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbe645d-d9cf-4a9d-9a19-8e89e5b5b1a3 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:37.507422 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 400/797] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Aug 24 17:08:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:08:37.519855 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbe645d-d9cf-4a9d-9a19-8e89e5b5b1a3 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:37.565871 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 398/798] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:08:37 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:08:37.566413 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:37.579570 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbccf172-48b8-4da6-85be-5a9e72592974 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:37.686478 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbccf172-48b8-4da6-85be-5a9e72592974 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:37.725194 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824198bf-ebe9-499a-b8ff-382c1a040c09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:37.725745 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824198bf-ebe9-499a-b8ff-382c1a040c09 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:37.726025 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824198bf-ebe9-499a-b8ff-382c1a040c09 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:37.726296 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824198bf-ebe9-499a-b8ff-382c1a040c09 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:37.739466 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824198bf-ebe9-499a-b8ff-382c1a040c09 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:08:37.743196 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-824198bf-ebe9-499a-b8ff-382c1a040c09 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:08:37.743196 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:37.743196 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:37.743196 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:37.743196 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:37.743196 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:37.743196 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:37.743196 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:37.743196 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:37.743196 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:37.743196 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:37.743196 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:37.743196 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:08:37.743196 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:08:37.743196 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:08:37.743196 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:08:37.744108 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:08:37.744108 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:08:37.744108 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:08:37.744108 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:08:37.744108 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:37.744108 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:37.744108 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:37.744108 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:37.744108 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:37.744108 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:37.744108 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:37.744108 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:37.744108 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:37.744108 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:37.744108 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:37.744108 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:37.744943 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:37.744943 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:37.744943 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:37.744943 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:37.744943 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:08:37.744943 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:08:37.744943 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:08:37.744943 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:08:37.744943 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:08:37.744943 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:08:37.744943 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:08:37.744943 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:08:37.747071 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbe645d-d9cf-4a9d-9a19-8e89e5b5b1a3 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:37.749045 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 399/799] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Aug 24 17:08:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:37.749828 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824198bf-ebe9-499a-b8ff-382c1a040c09 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:37.783221 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2544912a-d6bf-4b2b-83c2-286bac7aaf3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:37.784272 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2544912a-d6bf-4b2b-83c2-286bac7aaf3b None admin] REQUEST_METHOD: `PUT` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:37.784811 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2544912a-d6bf-4b2b-83c2-286bac7aaf3b None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:37.785306 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2544912a-d6bf-4b2b-83c2-286bac7aaf3b None admin] PATH_INFO: `/v3/projects/46ab158bc3644c888557781b0630b0b8/users/ef8caa353be34499a58cd25b72ee4032/roles/c5346bc54abf445f9900edf94264c850` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:37.787479 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2544912a-d6bf-4b2b-83c2-286bac7aaf3b None admin] RBAC: Authorizing `identity:create_grant(project_id=46ab158bc3644c888557781b0630b0b8, user_id=ef8caa353be34499a58cd25b72ee4032, role_id=c5346bc54abf445f9900edf94264c850)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:37.789153 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2544912a-d6bf-4b2b-83c2-286bac7aaf3b None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:37.812908 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:37.855223 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824198bf-ebe9-499a-b8ff-382c1a040c09 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:37.859062 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 401/800] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Aug 24 17:08:37 2022] PUT /identity/v3/projects/46ab158bc3644c888557781b0630b0b8/users/ef8caa353be34499a58cd25b72ee4032/roles/c5346bc54abf445f9900edf94264c850 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:08:37.867949 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2544912a-d6bf-4b2b-83c2-286bac7aaf3b None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:37.881687 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:37.896453 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30659c14-7f92-4991-be65-ec6d3ccaae21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:37.897488 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30659c14-7f92-4991-be65-ec6d3ccaae21 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:37.898089 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30659c14-7f92-4991-be65-ec6d3ccaae21 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:37.898713 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30659c14-7f92-4991-be65-ec6d3ccaae21 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:37.911490 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30659c14-7f92-4991-be65-ec6d3ccaae21 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:08:37.917074 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-30659c14-7f92-4991-be65-ec6d3ccaae21 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:08:37.917074 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:37.917074 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:37.917074 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:37.917074 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:37.917074 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:37.917074 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:37.917074 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:37.917074 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:37.917074 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:37.917074 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:37.917074 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:37.917074 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:08:37.917074 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:08:37.917074 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:08:37.917074 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:08:37.918632 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:08:37.918632 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:08:37.918632 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:08:37.918632 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:08:37.918632 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:37.918632 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:37.918632 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:37.918632 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:37.918632 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:37.918632 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:37.918632 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:37.918632 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:37.918632 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:37.918632 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:37.918632 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:37.918632 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:37.919644 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:37.919644 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:37.919644 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:37.919644 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:37.919644 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:08:37.919644 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:08:37.919644 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:08:37.919644 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:08:37.919644 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:08:37.919644 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:08:37.919644 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:08:37.919644 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:08:37.922325 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 400/801] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Aug 24 17:08:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:37.923464 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30659c14-7f92-4991-be65-ec6d3ccaae21 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:38.004327 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2544912a-d6bf-4b2b-83c2-286bac7aaf3b None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:38.047494 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bc7fcc-7257-4683-a6a0-61bde926ee38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:38.048175 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bc7fcc-7257-4683-a6a0-61bde926ee38 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:38.048463 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bc7fcc-7257-4683-a6a0-61bde926ee38 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:38.048745 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bc7fcc-7257-4683-a6a0-61bde926ee38 None admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:38.049406 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bc7fcc-7257-4683-a6a0-61bde926ee38 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:38.049713 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bc7fcc-7257-4683-a6a0-61bde926ee38 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:38.051832 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bc7fcc-7257-4683-a6a0-61bde926ee38 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:38.057365 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30659c14-7f92-4991-be65-ec6d3ccaae21 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:38.063153 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 402/802] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 24 17:08:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:38.068245 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bc7fcc-7257-4683-a6a0-61bde926ee38 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:38.087312 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:38.106950 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74639914-98d9-4e51-8d4a-71a146dd1241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:38.107586 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74639914-98d9-4e51-8d4a-71a146dd1241 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:38.107922 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74639914-98d9-4e51-8d4a-71a146dd1241 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:38.108204 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74639914-98d9-4e51-8d4a-71a146dd1241 None admin] PATH_INFO: `/v3/services/network` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:38.109706 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74639914-98d9-4e51-8d4a-71a146dd1241 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:38.111106 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74639914-98d9-4e51-8d4a-71a146dd1241 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:38.122890 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-74639914-98d9-4e51-8d4a-71a146dd1241 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 17:08:38.122890 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:38.122890 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 17:08:38.122890 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 17:08:38.122890 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 17:08:38.122890 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 17:08:38.122890 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 17:08:38.122890 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:08:38.122890 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 17:08:38.122890 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:08:38.122890 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 17:08:38.122890 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:08:38.122890 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:38.122890 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:38.122890 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:38.122890 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:38.122890 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:38.123949 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:38.123949 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:38.123949 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:38.123949 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:38.123949 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:38.123949 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:38.123949 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 17:08:38.123949 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 17:08:38.123949 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 17:08:38.123949 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 17:08:38.123949 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:38.123949 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:38.123949 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:38.123949 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:38.123949 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:38.123949 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:38.124832 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:38.124832 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:38.124832 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:38.124832 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:38.124832 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:38.124832 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:38.124832 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:38.124832 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:38.124832 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:38.124832 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:38.124832 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 17:08:38.124832 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:08:38.124832 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 17:08:38.124832 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:08:38.124832 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 401/803] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 24 17:08:37 2022] GET /identity/v3/services/network => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:38.125658 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74639914-98d9-4e51-8d4a-71a146dd1241 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:38.151434 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:38.189509 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bc7fcc-7257-4683-a6a0-61bde926ee38 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:38.229547 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d1b304-de29-4f07-8e87-8a540c9465c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:38.231213 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d1b304-de29-4f07-8e87-8a540c9465c7 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:38.231599 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d1b304-de29-4f07-8e87-8a540c9465c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:38.231982 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d1b304-de29-4f07-8e87-8a540c9465c7 None admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:38.232830 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d1b304-de29-4f07-8e87-8a540c9465c7 None admin] RBAC: Adding query filter params (name=service) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:38.233232 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d1b304-de29-4f07-8e87-8a540c9465c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:38.234713 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d1b304-de29-4f07-8e87-8a540c9465c7 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:38.244063 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 403/804] 10.222.0.18 () {56 vars in 1103 bytes} [Wed Aug 24 17:08:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:38.245029 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d1b304-de29-4f07-8e87-8a540c9465c7 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:38.259634 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74639914-98d9-4e51-8d4a-71a146dd1241 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:38.297680 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce67d46-f0fd-47da-83ab-a7a7ddc8eea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:38.298591 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce67d46-f0fd-47da-83ab-a7a7ddc8eea5 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:38.299055 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce67d46-f0fd-47da-83ab-a7a7ddc8eea5 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:38.299477 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce67d46-f0fd-47da-83ab-a7a7ddc8eea5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:38.310730 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce67d46-f0fd-47da-83ab-a7a7ddc8eea5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:38.312486 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce67d46-f0fd-47da-83ab-a7a7ddc8eea5 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:38.328450 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-dce67d46-f0fd-47da-83ab-a7a7ddc8eea5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:08:38.328450 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:38.328450 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:38.328450 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:38.328450 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:38.328450 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:38.328450 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:38.328450 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:38.328450 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:38.328450 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:38.328450 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:38.328450 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:38.328450 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:08:38.328450 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:08:38.328450 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:08:38.328450 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:08:38.329648 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:38.329648 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:38.329648 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:38.329648 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:38.329648 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:38.329648 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:38.329648 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:38.329648 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:38.329648 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:38.329648 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:38.329648 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:38.329648 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:38.329648 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:38.329648 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:38.329648 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:38.329648 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:38.329648 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:08:38.330552 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:08:38.330552 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:08:38.330552 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:08:38.337204 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 402/805] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 24 17:08:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:38.338327 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce67d46-f0fd-47da-83ab-a7a7ddc8eea5 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:38.353227 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d1b304-de29-4f07-8e87-8a540c9465c7 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:38.375129 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:38.387286 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:38.395041 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8e493b-64bf-4dff-8e2d-017e5ed77397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:38.396258 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8e493b-64bf-4dff-8e2d-017e5ed77397 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:38.396920 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8e493b-64bf-4dff-8e2d-017e5ed77397 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:38.397414 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8e493b-64bf-4dff-8e2d-017e5ed77397 None admin] PATH_INFO: `/v3/services` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:38.398537 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8e493b-64bf-4dff-8e2d-017e5ed77397 None admin] RBAC: Adding query filter params (name=network) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:38.399156 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8e493b-64bf-4dff-8e2d-017e5ed77397 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:38.401528 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8e493b-64bf-4dff-8e2d-017e5ed77397 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:38.414054 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 404/806] 10.222.0.18 () {56 vars in 1112 bytes} [Wed Aug 24 17:08:38 2022] GET /identity/v3/services?name=network => generated 124 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:38.414054 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8e493b-64bf-4dff-8e2d-017e5ed77397 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:38.531452 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8e493b-64bf-4dff-8e2d-017e5ed77397 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:38.574957 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb979b62-bf5e-4302-a3c9-831823c3be62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:38.575571 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb979b62-bf5e-4302-a3c9-831823c3be62 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:38.575852 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb979b62-bf5e-4302-a3c9-831823c3be62 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:38.576124 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb979b62-bf5e-4302-a3c9-831823c3be62 None admin] PATH_INFO: `/v3/domains` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:38.576771 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb979b62-bf5e-4302-a3c9-831823c3be62 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:38.577076 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb979b62-bf5e-4302-a3c9-831823c3be62 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:38.578441 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb979b62-bf5e-4302-a3c9-831823c3be62 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:38.591427 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 405/807] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 24 17:08:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:38.596362 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb979b62-bf5e-4302-a3c9-831823c3be62 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:38.613120 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce67d46-f0fd-47da-83ab-a7a7ddc8eea5 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:38.649350 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45cdb6d-786f-4fba-8b68-749c0860bb79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:38.649977 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45cdb6d-786f-4fba-8b68-749c0860bb79 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:38.650259 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45cdb6d-786f-4fba-8b68-749c0860bb79 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:38.650532 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45cdb6d-786f-4fba-8b68-749c0860bb79 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:38.651462 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:38.662614 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45cdb6d-786f-4fba-8b68-749c0860bb79 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:38.664079 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45cdb6d-786f-4fba-8b68-749c0860bb79 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:38.683704 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-e45cdb6d-786f-4fba-8b68-749c0860bb79 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:08:38.683704 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:38.683704 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:38.683704 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:38.683704 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:38.683704 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:38.683704 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:38.683704 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:38.683704 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:38.683704 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:38.683704 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:38.683704 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:38.683704 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:08:38.683704 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:08:38.683704 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:08:38.683704 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:08:38.684794 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:38.684794 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:38.684794 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:38.684794 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:38.684794 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:38.684794 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:38.684794 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:38.684794 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:38.684794 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:38.684794 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:38.684794 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:38.684794 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:38.684794 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:38.684794 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:38.684794 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:38.684794 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:38.684794 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:08:38.685709 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:08:38.685709 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:08:38.685709 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:08:38.685709 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 403/808] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 24 17:08:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 347 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:38.686365 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8b562d-2182-4304-9e49-1c9995b074ec None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:38.686471 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8b562d-2182-4304-9e49-1c9995b074ec None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:38.686667 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8b562d-2182-4304-9e49-1c9995b074ec None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:38.688147 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 404/809] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:08:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:08:38.688781 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8b562d-2182-4304-9e49-1c9995b074ec None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:38.702474 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb979b62-bf5e-4302-a3c9-831823c3be62 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:38.737502 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8623a19-86e4-494a-a11e-e3b5f0e5f3ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:38.738136 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8623a19-86e4-494a-a11e-e3b5f0e5f3ca None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:38.738413 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8623a19-86e4-494a-a11e-e3b5f0e5f3ca None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:38.738686 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8623a19-86e4-494a-a11e-e3b5f0e5f3ca None admin] PATH_INFO: `/v3/services` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:38.739334 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8623a19-86e4-494a-a11e-e3b5f0e5f3ca None admin] RBAC: Adding query filter params (type=network) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:38.739643 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8623a19-86e4-494a-a11e-e3b5f0e5f3ca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:38.741042 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8623a19-86e4-494a-a11e-e3b5f0e5f3ca None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:38.745814 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 406/810] 10.222.0.18 () {56 vars in 1112 bytes} [Wed Aug 24 17:08:38 2022] GET /identity/v3/services?type=network => generated 350 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:38.747262 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8623a19-86e4-494a-a11e-e3b5f0e5f3ca None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:38.784585 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:38.801510 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8b562d-2182-4304-9e49-1c9995b074ec None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:38.839189 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61860778-cf21-438a-8397-27ebdebfc590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:38.839906 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61860778-cf21-438a-8397-27ebdebfc590 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:38.840195 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61860778-cf21-438a-8397-27ebdebfc590 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:38.840525 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61860778-cf21-438a-8397-27ebdebfc590 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:38.848283 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61860778-cf21-438a-8397-27ebdebfc590 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:38.849796 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61860778-cf21-438a-8397-27ebdebfc590 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:38.850257 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8623a19-86e4-494a-a11e-e3b5f0e5f3ca None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:38.862106 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-61860778-cf21-438a-8397-27ebdebfc590 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 17:08:38.862106 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:38.862106 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:38.862106 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:38.862106 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:38.862106 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:38.862106 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:38.862106 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:38.862106 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:38.862106 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:38.862106 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:38.862106 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:38.862106 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:08:38.862106 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:08:38.862106 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:08:38.862106 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:08:38.863461 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:38.863461 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:38.863461 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:08:38.863461 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:08:38.863461 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:08:38.863461 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:08:38.863461 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:38.863461 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:38.863461 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:38.863461 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:38.863461 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:38.863461 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:38.863461 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:38.863461 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:38.863461 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:38.863461 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:38.863461 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:38.863461 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:38.864407 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:38.864407 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:38.864407 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:08:38.864407 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:08:38.864407 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:08:38.864407 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:08:38.864407 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:08:38.864407 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:08:38.864407 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 17:08:38.864407 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:08:38.864407 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 405/811] 10.222.0.18 () {56 vars in 1093 bytes} [Wed Aug 24 17:08:38 2022] GET /identity/v3/users/nova => generated 82 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:38.865027 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883415d4-046f-476e-82a1-bcf876fed58c None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:38.865159 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883415d4-046f-476e-82a1-bcf876fed58c None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:38.865420 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883415d4-046f-476e-82a1-bcf876fed58c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:38.872570 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:38.884751 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-883415d4-046f-476e-82a1-bcf876fed58c None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:08:38.885935 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-883415d4-046f-476e-82a1-bcf876fed58c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:08:38.889527 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f0bbe4-0c14-4893-9a5d-999a37d26b39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:38.890136 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f0bbe4-0c14-4893-9a5d-999a37d26b39 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:38.890411 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f0bbe4-0c14-4893-9a5d-999a37d26b39 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:38.890679 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f0bbe4-0c14-4893-9a5d-999a37d26b39 None admin] PATH_INFO: `/v3/domains` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:38.891348 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f0bbe4-0c14-4893-9a5d-999a37d26b39 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:38.891649 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f0bbe4-0c14-4893-9a5d-999a37d26b39 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:38.892993 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f0bbe4-0c14-4893-9a5d-999a37d26b39 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:38.909221 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 407/812] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 24 17:08:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:38.910665 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f0bbe4-0c14-4893-9a5d-999a37d26b39 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:38.970927 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 406/813] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:08:38 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:08:38.970927 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883415d4-046f-476e-82a1-bcf876fed58c None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:39.038590 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f0bbe4-0c14-4893-9a5d-999a37d26b39 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:39.071911 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:39.083452 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a586455e-2010-4518-8d9d-c9b9521a737e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:39.084216 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a586455e-2010-4518-8d9d-c9b9521a737e None admin] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:39.084570 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a586455e-2010-4518-8d9d-c9b9521a737e None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:39.084916 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a586455e-2010-4518-8d9d-c9b9521a737e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:39.085747 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a586455e-2010-4518-8d9d-c9b9521a737e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:39.087124 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a586455e-2010-4518-8d9d-c9b9521a737e None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:39.105682 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883415d4-046f-476e-82a1-bcf876fed58c None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:39.144252 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc376de1-09ba-499d-80fc-7635d5d32c43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:39.145107 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc376de1-09ba-499d-80fc-7635d5d32c43 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:39.145539 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc376de1-09ba-499d-80fc-7635d5d32c43 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:39.146104 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc376de1-09ba-499d-80fc-7635d5d32c43 None admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:39.147025 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc376de1-09ba-499d-80fc-7635d5d32c43 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:39.147508 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc376de1-09ba-499d-80fc-7635d5d32c43 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:39.149139 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc376de1-09ba-499d-80fc-7635d5d32c43 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:39.166935 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 407/814] 10.222.0.18 () {56 vars in 1133 bytes} [Wed Aug 24 17:08:38 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:39.167882 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc376de1-09ba-499d-80fc-7635d5d32c43 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:39.219894 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 408/815] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 24 17:08:38 2022] POST /identity/v3/endpoints => generated 337 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:08:39.221743 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a586455e-2010-4518-8d9d-c9b9521a737e None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:39.283316 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:39.322460 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc376de1-09ba-499d-80fc-7635d5d32c43 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:39.365585 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:39.378209 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a586455e-2010-4518-8d9d-c9b9521a737e None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:39.378891 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25813cd6-6ea1-44f0-b2f3-f93706c7bfcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:39.379757 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25813cd6-6ea1-44f0-b2f3-f93706c7bfcd None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:39.380201 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25813cd6-6ea1-44f0-b2f3-f93706c7bfcd None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:39.380682 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25813cd6-6ea1-44f0-b2f3-f93706c7bfcd None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:39.388758 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25813cd6-6ea1-44f0-b2f3-f93706c7bfcd None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:39.390552 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25813cd6-6ea1-44f0-b2f3-f93706c7bfcd None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:39.403184 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-25813cd6-6ea1-44f0-b2f3-f93706c7bfcd None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 17:08:39.403184 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:39.403184 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:39.403184 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:39.403184 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:39.403184 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:39.403184 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:39.403184 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:39.403184 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:39.403184 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:39.403184 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:39.403184 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:39.403184 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:08:39.403184 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:08:39.403184 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:08:39.403184 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:08:39.404356 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:39.404356 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:39.404356 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:08:39.404356 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:08:39.404356 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:08:39.404356 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:08:39.404356 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:39.404356 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:39.404356 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:39.404356 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:39.404356 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:39.404356 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:39.404356 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:39.404356 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:39.404356 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:39.404356 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:39.404356 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:39.404356 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:39.405353 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:39.405353 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:39.405353 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:08:39.405353 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:08:39.405353 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:08:39.405353 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:08:39.405353 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:08:39.405353 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:08:39.405353 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 17:08:39.405353 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:08:39.410363 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 408/816] 10.222.0.18 () {56 vars in 1117 bytes} [Wed Aug 24 17:08:39 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:39.411502 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25813cd6-6ea1-44f0-b2f3-f93706c7bfcd None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:39.427750 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921e13c6-15eb-4222-a669-b45a33cd597d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:39.428259 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921e13c6-15eb-4222-a669-b45a33cd597d None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:39.428538 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921e13c6-15eb-4222-a669-b45a33cd597d None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:39.428805 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921e13c6-15eb-4222-a669-b45a33cd597d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:39.508527 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921e13c6-15eb-4222-a669-b45a33cd597d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:39.509994 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921e13c6-15eb-4222-a669-b45a33cd597d None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:39.538383 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:39.539423 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25813cd6-6ea1-44f0-b2f3-f93706c7bfcd None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:39.579488 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061173af-caf7-4139-bd67-3f1d9b19b57e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:39.580356 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061173af-caf7-4139-bd67-3f1d9b19b57e None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:39.580785 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061173af-caf7-4139-bd67-3f1d9b19b57e None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:39.581215 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061173af-caf7-4139-bd67-3f1d9b19b57e None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:39.581794 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061173af-caf7-4139-bd67-3f1d9b19b57e None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:39.583267 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061173af-caf7-4139-bd67-3f1d9b19b57e None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:39.592411 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-061173af-caf7-4139-bd67-3f1d9b19b57e None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 17:08:39.592411 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:39.592411 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 17:08:39.592411 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 17:08:39.592411 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 17:08:39.592411 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 17:08:39.592411 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 17:08:39.592411 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:08:39.592411 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 17:08:39.592411 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:08:39.592411 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 17:08:39.592411 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:08:39.592411 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:39.592411 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:39.592411 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:39.592411 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:39.592411 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:39.593377 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:39.593377 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:39.593377 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:39.593377 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:39.593377 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:39.593377 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:39.593377 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 17:08:39.593377 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 17:08:39.593377 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 17:08:39.593377 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 17:08:39.593377 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:39.593377 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:39.593377 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:39.593377 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:39.593377 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:39.593377 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:39.594244 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:39.594244 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:39.594244 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:39.594244 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:39.594244 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:39.594244 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:39.594244 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:39.594244 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:39.594244 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:39.594244 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:39.594244 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 17:08:39.594244 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:08:39.594244 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 17:08:39.594244 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:08:39.594244 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 409/817] 10.222.0.18 () {56 vars in 1129 bytes} [Wed Aug 24 17:08:39 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:39.595078 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061173af-caf7-4139-bd67-3f1d9b19b57e None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:39.624817 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 409/818] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 24 17:08:39 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 17:08:39.626224 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921e13c6-15eb-4222-a669-b45a33cd597d None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:39.650067 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:39.703594 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061173af-caf7-4139-bd67-3f1d9b19b57e None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:39.730166 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921e13c6-15eb-4222-a669-b45a33cd597d None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:39.739321 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba2b253-03a3-492b-a7e2-d7cfd572cd00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:39.739929 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba2b253-03a3-492b-a7e2-d7cfd572cd00 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:39.740206 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba2b253-03a3-492b-a7e2-d7cfd572cd00 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:39.740478 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba2b253-03a3-492b-a7e2-d7cfd572cd00 None admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:39.741207 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba2b253-03a3-492b-a7e2-d7cfd572cd00 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:39.741510 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba2b253-03a3-492b-a7e2-d7cfd572cd00 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:39.742933 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba2b253-03a3-492b-a7e2-d7cfd572cd00 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:39.763463 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 410/819] 10.222.0.18 () {56 vars in 1149 bytes} [Wed Aug 24 17:08:39 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:39.764299 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba2b253-03a3-492b-a7e2-d7cfd572cd00 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:39.777282 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:39.783835 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56647d4e-21a9-42e7-b7a9-1754499fd40d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:39.784449 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56647d4e-21a9-42e7-b7a9-1754499fd40d None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:39.784732 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56647d4e-21a9-42e7-b7a9-1754499fd40d None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:39.785008 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56647d4e-21a9-42e7-b7a9-1754499fd40d None admin] PATH_INFO: `/v3/services` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:39.785644 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56647d4e-21a9-42e7-b7a9-1754499fd40d None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:39.785948 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56647d4e-21a9-42e7-b7a9-1754499fd40d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:39.787355 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56647d4e-21a9-42e7-b7a9-1754499fd40d None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:39.791795 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 410/820] 10.222.0.18 () {56 vars in 1124 bytes} [Wed Aug 24 17:08:39 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:39.792624 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56647d4e-21a9-42e7-b7a9-1754499fd40d None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:39.874731 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba2b253-03a3-492b-a7e2-d7cfd572cd00 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:39.885896 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:39.897610 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56647d4e-21a9-42e7-b7a9-1754499fd40d None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:39.908956 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fa2110-5c51-4572-9cd9-7592f16c1cb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:39.909584 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fa2110-5c51-4572-9cd9-7592f16c1cb2 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:39.909865 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fa2110-5c51-4572-9cd9-7592f16c1cb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:39.910143 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fa2110-5c51-4572-9cd9-7592f16c1cb2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:39.920831 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fa2110-5c51-4572-9cd9-7592f16c1cb2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:39.922012 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fa2110-5c51-4572-9cd9-7592f16c1cb2 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:39.930918 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9322ff4e-f2a7-422e-bde4-41e731d74318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:39.931549 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9322ff4e-f2a7-422e-bde4-41e731d74318 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:39.931830 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9322ff4e-f2a7-422e-bde4-41e731d74318 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:39.932103 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9322ff4e-f2a7-422e-bde4-41e731d74318 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:39.937323 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-27fa2110-5c51-4572-9cd9-7592f16c1cb2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:08:39.937323 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:39.937323 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:39.937323 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:39.937323 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:39.937323 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:39.937323 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:39.937323 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:39.937323 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:39.937323 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:39.937323 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:39.937323 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:39.937323 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:08:39.937323 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:08:39.937323 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:08:39.937323 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:08:39.938423 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:39.938423 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:39.938423 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:39.938423 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:39.938423 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:39.938423 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:39.938423 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:39.938423 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:39.938423 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:39.938423 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:39.938423 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:39.938423 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:39.938423 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:39.938423 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:39.938423 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:39.938423 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:39.938423 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:08:39.939699 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:08:39.939699 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:08:39.939699 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:08:39.939699 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 411/821] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 24 17:08:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:39.939699 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fa2110-5c51-4572-9cd9-7592f16c1cb2 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:39.944111 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9322ff4e-f2a7-422e-bde4-41e731d74318 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:39.945551 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9322ff4e-f2a7-422e-bde4-41e731d74318 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:39.959430 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-9322ff4e-f2a7-422e-bde4-41e731d74318 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:08:39.959430 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:39.959430 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:39.959430 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:39.959430 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:39.959430 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:39.959430 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:39.959430 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:39.959430 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:39.959430 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:39.959430 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:39.959430 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:39.959430 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:08:39.959430 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:08:39.959430 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:08:39.959430 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:08:39.960481 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:39.960481 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:39.960481 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:39.960481 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:39.960481 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:39.960481 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:39.960481 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:39.960481 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:39.960481 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:39.960481 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:39.960481 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:39.960481 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:39.960481 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:39.960481 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:39.960481 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:39.960481 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:39.960481 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:08:39.961380 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:08:39.961380 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:08:39.961380 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:08:39.961380 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 411/822] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 24 17:08:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:39.962048 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7861f2-8efb-42be-b7b0-86e7d091c735 None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:39.962048 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7861f2-8efb-42be-b7b0-86e7d091c735 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:39.962206 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7861f2-8efb-42be-b7b0-86e7d091c735 None None] PATH_INFO: `/` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:39.963374 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 412/823] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:08:39 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:08:39.964053 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7861f2-8efb-42be-b7b0-86e7d091c735 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:40.011020 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:40.045911 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fa2110-5c51-4572-9cd9-7592f16c1cb2 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:40.064291 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7861f2-8efb-42be-b7b0-86e7d091c735 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:40.078540 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c2423e-5296-4141-b900-738ea58621c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:40.079165 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c2423e-5296-4141-b900-738ea58621c0 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:40.079447 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c2423e-5296-4141-b900-738ea58621c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:40.079719 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c2423e-5296-4141-b900-738ea58621c0 None admin] PATH_INFO: `/v3/services` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:40.080337 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c2423e-5296-4141-b900-738ea58621c0 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:40.080635 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c2423e-5296-4141-b900-738ea58621c0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:40.081992 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c2423e-5296-4141-b900-738ea58621c0 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:40.086613 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 412/824] 10.222.0.18 () {56 vars in 1124 bytes} [Wed Aug 24 17:08:39 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:40.087534 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c2423e-5296-4141-b900-738ea58621c0 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:40.096769 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c2b41c-c730-4fab-abee-58aa329e39d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:40.097383 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c2b41c-c730-4fab-abee-58aa329e39d9 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:40.097725 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c2b41c-c730-4fab-abee-58aa329e39d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:40.098004 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c2b41c-c730-4fab-abee-58aa329e39d9 None admin] PATH_INFO: `/v3/domains` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:40.098644 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c2b41c-c730-4fab-abee-58aa329e39d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:40.098978 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c2b41c-c730-4fab-abee-58aa329e39d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:40.100351 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c2b41c-c730-4fab-abee-58aa329e39d9 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:40.113141 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 413/825] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 24 17:08:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:40.114293 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a659f1ae-0df1-49e8-84ab-2f6f16ec4c22 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:40.114584 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a659f1ae-0df1-49e8-84ab-2f6f16ec4c22 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:40.115054 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a659f1ae-0df1-49e8-84ab-2f6f16ec4c22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:40.117344 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:40.132527 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-a659f1ae-0df1-49e8-84ab-2f6f16ec4c22 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:08:40.133643 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-a659f1ae-0df1-49e8-84ab-2f6f16ec4c22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:08:40.190092 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c2423e-5296-4141-b900-738ea58621c0 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:40.197770 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 414/826] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:08:40 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:08:40.198566 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a659f1ae-0df1-49e8-84ab-2f6f16ec4c22 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:40.231485 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ae5be9-e4fe-4ddd-aabb-d8df80088e57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:40.232087 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ae5be9-e4fe-4ddd-aabb-d8df80088e57 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:40.232363 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ae5be9-e4fe-4ddd-aabb-d8df80088e57 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:40.232744 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ae5be9-e4fe-4ddd-aabb-d8df80088e57 None admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:40.233382 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ae5be9-e4fe-4ddd-aabb-d8df80088e57 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:40.233684 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ae5be9-e4fe-4ddd-aabb-d8df80088e57 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:40.234867 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:40.235074 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ae5be9-e4fe-4ddd-aabb-d8df80088e57 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:40.248031 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 413/827] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 24 17:08:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:40.248920 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ae5be9-e4fe-4ddd-aabb-d8df80088e57 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:40.311956 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a659f1ae-0df1-49e8-84ab-2f6f16ec4c22 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:40.341483 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:40.351485 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7efd2c5-03c7-4c72-92a2-8160ac1efee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:40.352023 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7efd2c5-03c7-4c72-92a2-8160ac1efee8 None admin] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:40.352460 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7efd2c5-03c7-4c72-92a2-8160ac1efee8 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:40.355319 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7efd2c5-03c7-4c72-92a2-8160ac1efee8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:40.355319 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7efd2c5-03c7-4c72-92a2-8160ac1efee8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:40.355319 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7efd2c5-03c7-4c72-92a2-8160ac1efee8 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:40.358464 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ae5be9-e4fe-4ddd-aabb-d8df80088e57 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:40.393021 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4450e7f3-ed83-448e-a4ac-0d5db49ba5de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:40.393632 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4450e7f3-ed83-448e-a4ac-0d5db49ba5de None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:40.394018 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4450e7f3-ed83-448e-a4ac-0d5db49ba5de None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:40.394290 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4450e7f3-ed83-448e-a4ac-0d5db49ba5de None admin] PATH_INFO: `/v3/projects/service` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:40.400883 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4450e7f3-ed83-448e-a4ac-0d5db49ba5de None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:40.403094 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4450e7f3-ed83-448e-a4ac-0d5db49ba5de None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:40.414125 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-4450e7f3-ed83-448e-a4ac-0d5db49ba5de None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:08:40.414125 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:40.414125 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:40.414125 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:40.414125 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:40.414125 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:40.414125 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:40.414125 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:40.414125 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:40.414125 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:40.414125 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:40.414125 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:40.414125 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:08:40.414125 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:08:40.414125 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:08:40.415256 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:08:40.415256 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:40.415256 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:40.415256 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:40.415256 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:40.415256 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:40.415256 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:40.415256 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:40.415256 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:40.415256 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:40.415256 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:40.415256 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:40.415256 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:40.415256 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:40.415256 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:40.415256 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:40.415256 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:40.416154 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:08:40.416154 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:08:40.416154 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:08:40.416154 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:08:40.416154 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:08:40.416154 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:08:40.416154 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:08:40.416154 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:08:40.416154 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 414/828] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 24 17:08:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:40.416154 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4450e7f3-ed83-448e-a4ac-0d5db49ba5de None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:40.458237 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 415/829] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 24 17:08:40 2022] POST /identity/v3/endpoints => generated 346 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:08:40.459262 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7efd2c5-03c7-4c72-92a2-8160ac1efee8 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:40.534462 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4450e7f3-ed83-448e-a4ac-0d5db49ba5de None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:40.571619 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:40.586606 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc61f8f8-781a-4862-9bac-f3ed787bfa10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:40.587172 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7efd2c5-03c7-4c72-92a2-8160ac1efee8 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:40.587793 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc61f8f8-781a-4862-9bac-f3ed787bfa10 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:40.587793 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc61f8f8-781a-4862-9bac-f3ed787bfa10 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:40.587982 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc61f8f8-781a-4862-9bac-f3ed787bfa10 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:40.596586 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc61f8f8-781a-4862-9bac-f3ed787bfa10 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:40.598711 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc61f8f8-781a-4862-9bac-f3ed787bfa10 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:40.611933 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-fc61f8f8-781a-4862-9bac-f3ed787bfa10 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:08:40.611933 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:40.611933 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:40.611933 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:40.611933 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:40.611933 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:40.611933 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:40.611933 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:40.611933 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:40.611933 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:40.611933 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:40.611933 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:40.611933 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:08:40.611933 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:08:40.611933 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:08:40.613041 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:08:40.613041 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:40.613041 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:40.613041 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:40.613041 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:40.613041 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:40.613041 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:40.613041 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:40.613041 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:40.613041 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:40.613041 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:40.613041 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:40.613041 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:40.613041 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:40.613041 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:40.613041 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:40.613041 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:40.614379 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:08:40.614379 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:08:40.614379 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:08:40.614379 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:08:40.614379 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:08:40.614379 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:08:40.614379 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:08:40.614379 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:08:40.614379 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 415/830] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 24 17:08:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:40.614379 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc61f8f8-781a-4862-9bac-f3ed787bfa10 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:40.619442 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:40.632062 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8330477b-7c86-4360-8f98-07b32b4fb5b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:40.633500 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8330477b-7c86-4360-8f98-07b32b4fb5b1 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:40.633997 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8330477b-7c86-4360-8f98-07b32b4fb5b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:40.634116 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8330477b-7c86-4360-8f98-07b32b4fb5b1 None admin] PATH_INFO: `/v3/projects` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:40.634762 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8330477b-7c86-4360-8f98-07b32b4fb5b1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:40.639114 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8330477b-7c86-4360-8f98-07b32b4fb5b1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:40.640580 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8330477b-7c86-4360-8f98-07b32b4fb5b1 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:40.654406 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 416/831] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Aug 24 17:08:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:40.655050 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8330477b-7c86-4360-8f98-07b32b4fb5b1 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:40.734463 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc61f8f8-781a-4862-9bac-f3ed787bfa10 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:40.758643 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8330477b-7c86-4360-8f98-07b32b4fb5b1 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:40.770171 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b205040-13f9-4e2d-9ef0-cfb9923aa192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:40.771047 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b205040-13f9-4e2d-9ef0-cfb9923aa192 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:40.771161 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b205040-13f9-4e2d-9ef0-cfb9923aa192 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:40.771366 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b205040-13f9-4e2d-9ef0-cfb9923aa192 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:40.782722 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b205040-13f9-4e2d-9ef0-cfb9923aa192 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:08:40.788722 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-7b205040-13f9-4e2d-9ef0-cfb9923aa192 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:08:40.788722 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:40.788722 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:40.788722 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:40.788722 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:40.788722 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:40.788722 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:40.788722 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:40.788722 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:40.788722 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:40.788722 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:40.788722 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:40.788722 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:08:40.788722 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:08:40.788722 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:08:40.788722 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:08:40.789663 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:08:40.789663 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:08:40.789663 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:08:40.789663 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:08:40.789663 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:40.789663 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:40.789663 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:40.789663 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:40.789663 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:40.789663 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:40.789663 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:40.789663 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:40.789663 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:40.789663 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:40.789663 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:40.789663 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:40.790498 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:40.790498 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:40.790498 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:40.790498 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:40.790498 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:08:40.790498 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:08:40.790498 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:08:40.790498 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:08:40.790498 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:08:40.790498 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:08:40.790498 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:08:40.790498 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:08:40.790498 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 416/832] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Aug 24 17:08:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:40.791242 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b205040-13f9-4e2d-9ef0-cfb9923aa192 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:40.801700 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f60f2b5-6d5d-429e-b164-fe64b269e8ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:40.802314 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f60f2b5-6d5d-429e-b164-fe64b269e8ba None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:40.802591 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f60f2b5-6d5d-429e-b164-fe64b269e8ba None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:40.802860 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f60f2b5-6d5d-429e-b164-fe64b269e8ba None admin] PATH_INFO: `/v3/projects` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:40.803584 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f60f2b5-6d5d-429e-b164-fe64b269e8ba None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:40.803883 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f60f2b5-6d5d-429e-b164-fe64b269e8ba None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:40.805289 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f60f2b5-6d5d-429e-b164-fe64b269e8ba None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:40.818083 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 417/833] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Aug 24 17:08:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:40.818915 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f60f2b5-6d5d-429e-b164-fe64b269e8ba None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:40.832114 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:40.861744 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:40.910799 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b205040-13f9-4e2d-9ef0-cfb9923aa192 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:40.926659 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f60f2b5-6d5d-429e-b164-fe64b269e8ba None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:40.950759 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e9cc77-016d-47dc-b554-86c727fa2d90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:40.951410 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e9cc77-016d-47dc-b554-86c727fa2d90 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:40.951695 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e9cc77-016d-47dc-b554-86c727fa2d90 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:40.951973 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e9cc77-016d-47dc-b554-86c727fa2d90 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:40.952709 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e9cc77-016d-47dc-b554-86c727fa2d90 None admin] RBAC: Adding query filter params (role.id=c5346bc54abf445f9900edf94264c850, scope.project.id=46ab158bc3644c888557781b0630b0b8, user.id=557aca9255cb4bbb8af52e796c5c5177) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:40.953020 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e9cc77-016d-47dc-b554-86c727fa2d90 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:40.954458 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e9cc77-016d-47dc-b554-86c727fa2d90 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:40.960507 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 417/834] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 24 17:08:40 2022] GET /identity/v3/role_assignments?user.id=557aca9255cb4bbb8af52e796c5c5177&scope.project.id=46ab158bc3644c888557781b0630b0b8&role.id=c5346bc54abf445f9900edf94264c850 => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:40.961264 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e9cc77-016d-47dc-b554-86c727fa2d90 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:40.983549 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f0ce07-9c1e-4b09-afa6-248fd07b6cf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:40.984430 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f0ce07-9c1e-4b09-afa6-248fd07b6cf3 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:40.984853 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f0ce07-9c1e-4b09-afa6-248fd07b6cf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:40.985291 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f0ce07-9c1e-4b09-afa6-248fd07b6cf3 None admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:40.986113 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f0ce07-9c1e-4b09-afa6-248fd07b6cf3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:40.986562 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f0ce07-9c1e-4b09-afa6-248fd07b6cf3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:40.988229 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f0ce07-9c1e-4b09-afa6-248fd07b6cf3 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:40.998317 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 418/835] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 24 17:08:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:41.005080 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f0ce07-9c1e-4b09-afa6-248fd07b6cf3 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:41.090936 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e9cc77-016d-47dc-b554-86c727fa2d90 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:41.122119 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:41.132314 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c5fa6a-8ff0-49fc-ab2c-77822ad1f4a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:41.133248 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c5fa6a-8ff0-49fc-ab2c-77822ad1f4a4 None admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:41.135998 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c5fa6a-8ff0-49fc-ab2c-77822ad1f4a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:41.135998 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c5fa6a-8ff0-49fc-ab2c-77822ad1f4a4 None admin] PATH_INFO: `/v3/projects/46ab158bc3644c888557781b0630b0b8/users/0ccc6723f7514e52863d102de3839951/roles/e7db2a847f56438bac13d0ff21c96b73` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:41.160702 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c5fa6a-8ff0-49fc-ab2c-77822ad1f4a4 None admin] RBAC: Authorizing `identity:create_grant(project_id=46ab158bc3644c888557781b0630b0b8, user_id=0ccc6723f7514e52863d102de3839951, role_id=e7db2a847f56438bac13d0ff21c96b73)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:41.163099 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c5fa6a-8ff0-49fc-ab2c-77822ad1f4a4 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:41.205574 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:41.236474 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 418/836] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Aug 24 17:08:40 2022] PUT /identity/v3/projects/46ab158bc3644c888557781b0630b0b8/users/0ccc6723f7514e52863d102de3839951/roles/e7db2a847f56438bac13d0ff21c96b73 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:08:41.241803 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f0ce07-9c1e-4b09-afa6-248fd07b6cf3 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:41.285011 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886d3f12-13e3-4e29-9878-6343a5aea573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:41.285935 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886d3f12-13e3-4e29-9878-6343a5aea573 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:41.286399 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886d3f12-13e3-4e29-9878-6343a5aea573 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:41.286834 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886d3f12-13e3-4e29-9878-6343a5aea573 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:41.298831 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886d3f12-13e3-4e29-9878-6343a5aea573 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:41.300602 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886d3f12-13e3-4e29-9878-6343a5aea573 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:41.317616 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-886d3f12-13e3-4e29-9878-6343a5aea573 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:08:41.317616 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:41.317616 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:41.317616 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:41.317616 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:41.317616 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:41.317616 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:41.317616 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:41.317616 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:41.317616 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:41.317616 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:41.317616 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:41.317616 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:08:41.317616 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:08:41.317616 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:08:41.317616 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:08:41.318852 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:41.318852 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:41.318852 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:41.318852 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:41.318852 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:41.318852 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:41.318852 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:41.318852 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:41.318852 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:41.318852 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:41.318852 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:41.318852 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:41.318852 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:41.318852 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:41.318852 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:41.318852 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:41.318852 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:08:41.319985 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:08:41.319985 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:08:41.319985 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:08:41.321849 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 419/837] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 24 17:08:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:08:41.328189 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c5fa6a-8ff0-49fc-ab2c-77822ad1f4a4 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:41.445793 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c5fa6a-8ff0-49fc-ab2c-77822ad1f4a4 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:41.482680 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c115a5-8d6c-4115-b242-b3c7d15a628c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:41.483705 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c115a5-8d6c-4115-b242-b3c7d15a628c None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:41.484178 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c115a5-8d6c-4115-b242-b3c7d15a628c None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:41.484643 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c115a5-8d6c-4115-b242-b3c7d15a628c None admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:41.485524 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c115a5-8d6c-4115-b242-b3c7d15a628c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:41.486029 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c115a5-8d6c-4115-b242-b3c7d15a628c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:41.487635 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c115a5-8d6c-4115-b242-b3c7d15a628c None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:41.500471 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 419/838] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 24 17:08:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:41.506812 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886d3f12-13e3-4e29-9878-6343a5aea573 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:41.616675 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886d3f12-13e3-4e29-9878-6343a5aea573 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:41.641476 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:41.653700 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2034385-109d-41df-a4c7-039a0ae8efa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:41.654326 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2034385-109d-41df-a4c7-039a0ae8efa0 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:41.654647 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2034385-109d-41df-a4c7-039a0ae8efa0 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:41.654941 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2034385-109d-41df-a4c7-039a0ae8efa0 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:41.662188 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2034385-109d-41df-a4c7-039a0ae8efa0 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:41.663636 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2034385-109d-41df-a4c7-039a0ae8efa0 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:41.674983 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-b2034385-109d-41df-a4c7-039a0ae8efa0 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 17:08:41.674983 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:41.674983 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:41.674983 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:41.674983 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:41.674983 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:41.674983 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:41.674983 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:41.674983 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:41.674983 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:41.674983 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:41.674983 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:41.674983 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:08:41.674983 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:08:41.674983 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:08:41.674983 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:08:41.676005 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:41.676005 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:41.676005 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:08:41.676005 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:08:41.676005 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:08:41.676005 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:08:41.676005 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:41.676005 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:41.676005 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:41.676005 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:41.676005 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:41.676005 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:41.676005 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:41.676005 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:41.676005 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:41.676005 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:41.676005 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:41.676005 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:41.676938 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:41.676938 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:41.676938 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:08:41.676938 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:08:41.676938 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:08:41.676938 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:08:41.676938 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:08:41.676938 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:08:41.676938 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 17:08:41.676938 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:08:41.676938 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 420/839] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Aug 24 17:08:41 2022] GET /identity/v3/users/swift => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:41.683408 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c115a5-8d6c-4115-b242-b3c7d15a628c None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:41.727254 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:41.811800 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c115a5-8d6c-4115-b242-b3c7d15a628c None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:41.846474 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76890ca5-13ad-4aca-adc5-b19834647b04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:41.847155 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76890ca5-13ad-4aca-adc5-b19834647b04 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:41.847435 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76890ca5-13ad-4aca-adc5-b19834647b04 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:41.847773 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76890ca5-13ad-4aca-adc5-b19834647b04 None admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:41.848639 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76890ca5-13ad-4aca-adc5-b19834647b04 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:41.848946 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76890ca5-13ad-4aca-adc5-b19834647b04 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:41.850357 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76890ca5-13ad-4aca-adc5-b19834647b04 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:41.868437 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 420/840] 10.222.0.18 () {56 vars in 1135 bytes} [Wed Aug 24 17:08:41 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:41.874623 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2034385-109d-41df-a4c7-039a0ae8efa0 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:41.980099 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2034385-109d-41df-a4c7-039a0ae8efa0 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:42.014457 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2276359-a2f8-4b26-8685-46ed3a719420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:42.015161 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2276359-a2f8-4b26-8685-46ed3a719420 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:42.015576 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2276359-a2f8-4b26-8685-46ed3a719420 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:42.015846 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2276359-a2f8-4b26-8685-46ed3a719420 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:42.066848 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:42.085201 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2276359-a2f8-4b26-8685-46ed3a719420 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:42.088801 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2276359-a2f8-4b26-8685-46ed3a719420 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:42.188810 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 421/841] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 24 17:08:41 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 17:08:42.194580 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76890ca5-13ad-4aca-adc5-b19834647b04 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:42.296768 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:42.304128 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76890ca5-13ad-4aca-adc5-b19834647b04 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:42.341750 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb5e89b-bece-4809-b164-3cc0e006095b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:42.342732 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb5e89b-bece-4809-b164-3cc0e006095b None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:42.343245 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb5e89b-bece-4809-b164-3cc0e006095b None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:42.353125 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb5e89b-bece-4809-b164-3cc0e006095b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:42.354611 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb5e89b-bece-4809-b164-3cc0e006095b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:42.356310 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb5e89b-bece-4809-b164-3cc0e006095b None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:42.371097 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-5fb5e89b-bece-4809-b164-3cc0e006095b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:08:42.371097 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:42.371097 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:42.371097 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:42.371097 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:42.371097 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:42.371097 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:42.371097 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:42.371097 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:42.371097 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:42.371097 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:42.371097 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:42.371097 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:08:42.371097 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:08:42.371097 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:08:42.371097 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:08:42.372057 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:42.372057 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:42.372057 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:42.372057 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:42.372057 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:42.372057 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:42.372057 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:42.372057 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:42.372057 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:42.372057 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:42.372057 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:42.372057 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:42.372057 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:42.372057 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:42.372057 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:42.372057 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:42.372057 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:08:42.372933 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:08:42.372933 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:08:42.372933 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:08:42.379062 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 421/842] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 24 17:08:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:42.381117 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2276359-a2f8-4b26-8685-46ed3a719420 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:42.474137 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751131f2-b39e-434d-ae5a-f7463fca40cd None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:42.474462 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751131f2-b39e-434d-ae5a-f7463fca40cd None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:42.474690 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751131f2-b39e-434d-ae5a-f7463fca40cd None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:42.476417 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 422/843] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:08:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 17:08:42.479249 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:42.482426 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad27b70-a240-4a56-a378-cb483592f69c None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:42.482711 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad27b70-a240-4a56-a378-cb483592f69c None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:42.483027 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad27b70-a240-4a56-a378-cb483592f69c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:42.502313 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-2ad27b70-a240-4a56-a378-cb483592f69c None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:08:42.503003 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2276359-a2f8-4b26-8685-46ed3a719420 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:42.503518 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-2ad27b70-a240-4a56-a378-cb483592f69c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:08:42.555494 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dfe167-9c2d-485e-b7dd-80c88ff1a78b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:42.556264 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dfe167-9c2d-485e-b7dd-80c88ff1a78b None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:42.556547 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dfe167-9c2d-485e-b7dd-80c88ff1a78b None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:42.556817 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dfe167-9c2d-485e-b7dd-80c88ff1a78b None admin] PATH_INFO: `/v3/domains` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:42.557468 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9dfe167-9c2d-485e-b7dd-80c88ff1a78b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:42.557766 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9dfe167-9c2d-485e-b7dd-80c88ff1a78b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:42.560488 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9dfe167-9c2d-485e-b7dd-80c88ff1a78b None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:42.576786 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 422/844] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 24 17:08:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:42.581831 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 423/845] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:08:42 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:08:42.582578 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dfe167-9c2d-485e-b7dd-80c88ff1a78b None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:42.690061 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dfe167-9c2d-485e-b7dd-80c88ff1a78b None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:42.725200 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc95899-910f-41c9-ac8a-ff2469284171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:42.726178 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc95899-910f-41c9-ac8a-ff2469284171 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:42.726669 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc95899-910f-41c9-ac8a-ff2469284171 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:42.727166 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc95899-910f-41c9-ac8a-ff2469284171 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:42.734284 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc95899-910f-41c9-ac8a-ff2469284171 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:42.735996 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc95899-910f-41c9-ac8a-ff2469284171 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:42.748013 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-9bc95899-910f-41c9-ac8a-ff2469284171 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:08:42.748013 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:42.748013 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:42.748013 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:42.748013 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:42.748013 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:42.748013 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:42.748013 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:42.748013 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:42.748013 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:42.748013 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:42.748013 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:42.748013 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:08:42.748013 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:08:42.748013 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:08:42.748912 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:08:42.748912 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:42.748912 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:42.748912 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:42.748912 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:42.748912 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:42.748912 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:42.748912 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:42.748912 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:42.748912 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:42.748912 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:42.748912 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:42.748912 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:42.748912 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:42.748912 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:42.748912 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:42.748912 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:42.749826 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:08:42.749826 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:08:42.749826 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:08:42.749826 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:08:42.749826 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:08:42.749826 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:08:42.749826 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:08:42.749826 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:08:42.752523 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 423/846] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 24 17:08:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:42.760389 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad27b70-a240-4a56-a378-cb483592f69c None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:42.841713 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:42.865276 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad27b70-a240-4a56-a378-cb483592f69c None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:42.882067 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc95899-910f-41c9-ac8a-ff2469284171 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:42.887502 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:42.903636 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4073197f-677d-489f-9459-7e79570fbb29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:42.904463 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4073197f-677d-489f-9459-7e79570fbb29 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:42.904884 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4073197f-677d-489f-9459-7e79570fbb29 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:42.905293 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4073197f-677d-489f-9459-7e79570fbb29 None admin] PATH_INFO: `/v3/projects` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:42.906142 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4073197f-677d-489f-9459-7e79570fbb29 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:42.906596 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4073197f-677d-489f-9459-7e79570fbb29 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:42.908180 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4073197f-677d-489f-9459-7e79570fbb29 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:42.926075 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 424/847] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Aug 24 17:08:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:42.927460 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4073197f-677d-489f-9459-7e79570fbb29 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:43.020122 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc95899-910f-41c9-ac8a-ff2469284171 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:43.036436 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4073197f-677d-489f-9459-7e79570fbb29 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:43.067068 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f2dcd9-7494-4386-bc70-70037657ea59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:43.067714 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f2dcd9-7494-4386-bc70-70037657ea59 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:43.067998 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f2dcd9-7494-4386-bc70-70037657ea59 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:43.068278 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f2dcd9-7494-4386-bc70-70037657ea59 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:43.069286 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f2dcd9-7494-4386-bc70-70037657ea59 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:43.070960 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f2dcd9-7494-4386-bc70-70037657ea59 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:43.074200 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cbefcf-b3af-4da8-8bda-4617e7dadef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:43.075163 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cbefcf-b3af-4da8-8bda-4617e7dadef4 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:43.075640 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cbefcf-b3af-4da8-8bda-4617e7dadef4 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:43.076103 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cbefcf-b3af-4da8-8bda-4617e7dadef4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:43.077031 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cbefcf-b3af-4da8-8bda-4617e7dadef4 None admin] RBAC: Adding query filter params (role.id=c5346bc54abf445f9900edf94264c850, scope.project.id=46ab158bc3644c888557781b0630b0b8, user.id=ef8caa353be34499a58cd25b72ee4032) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:43.077547 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cbefcf-b3af-4da8-8bda-4617e7dadef4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:43.079179 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cbefcf-b3af-4da8-8bda-4617e7dadef4 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:43.079983 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-43f2dcd9-7494-4386-bc70-70037657ea59 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 17:08:43.079983 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:43.079983 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 17:08:43.079983 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 17:08:43.079983 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 17:08:43.079983 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 17:08:43.079983 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 17:08:43.079983 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:08:43.079983 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 17:08:43.079983 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:08:43.079983 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 17:08:43.079983 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:08:43.079983 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:43.079983 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:43.079983 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:43.079983 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:43.079983 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:43.080958 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:43.080958 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:43.080958 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:43.080958 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:43.080958 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:43.080958 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:43.080958 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 17:08:43.080958 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 17:08:43.080958 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 17:08:43.080958 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 17:08:43.080958 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:43.080958 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:43.080958 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:43.080958 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:43.080958 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:43.080958 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:43.081803 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:43.081803 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:43.081803 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:43.081803 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:43.081803 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:43.081803 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:43.081803 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:43.081803 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:43.081803 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:43.081803 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:43.081803 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 17:08:43.081803 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:08:43.081803 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 17:08:43.081803 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:08:43.081803 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 424/848] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Aug 24 17:08:42 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:43.087909 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:43.088763 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f2dcd9-7494-4386-bc70-70037657ea59 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:43.091268 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 425/849] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 24 17:08:42 2022] GET /identity/v3/role_assignments?user.id=ef8caa353be34499a58cd25b72ee4032&scope.project.id=46ab158bc3644c888557781b0630b0b8&role.id=c5346bc54abf445f9900edf94264c850 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:08:43.124367 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c2ac3e-011b-44d0-8c7a-336e9cafc2fe None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:43.124691 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c2ac3e-011b-44d0-8c7a-336e9cafc2fe None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:43.124901 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c2ac3e-011b-44d0-8c7a-336e9cafc2fe None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:43.127059 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 426/850] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:08:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 17:08:43.133310 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d7e3f3-db95-4f6a-8027-d473344a28b7 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:43.133993 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d7e3f3-db95-4f6a-8027-d473344a28b7 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:43.134468 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d7e3f3-db95-4f6a-8027-d473344a28b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:43.154519 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-e2d7e3f3-db95-4f6a-8027-d473344a28b7 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:08:43.156575 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-e2d7e3f3-db95-4f6a-8027-d473344a28b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:08:43.177223 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:43.227058 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f2dcd9-7494-4386-bc70-70037657ea59 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:43.253382 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 427/851] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:08:43 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 17:08:43.260310 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f9712f-d7d8-474c-a39a-f9ff778a7ab6 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:43.260820 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f9712f-d7d8-474c-a39a-f9ff778a7ab6 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:43.261256 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f9712f-d7d8-474c-a39a-f9ff778a7ab6 None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:43.263630 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 428/852] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:08:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 17:08:43.268717 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3076109d-3672-439d-b701-d9b87a044438 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:43.269003 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3076109d-3672-439d-b701-d9b87a044438 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:43.269277 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3076109d-3672-439d-b701-d9b87a044438 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:43.279760 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a868fe4-e0bb-42e3-ba3b-88872e1a72c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:43.280378 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a868fe4-e0bb-42e3-ba3b-88872e1a72c0 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:43.280658 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a868fe4-e0bb-42e3-ba3b-88872e1a72c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:43.280937 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a868fe4-e0bb-42e3-ba3b-88872e1a72c0 None admin] PATH_INFO: `/v3/services` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:43.281597 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a868fe4-e0bb-42e3-ba3b-88872e1a72c0 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:43.281902 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a868fe4-e0bb-42e3-ba3b-88872e1a72c0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:43.283662 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a868fe4-e0bb-42e3-ba3b-88872e1a72c0 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:43.288211 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 425/853] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Aug 24 17:08:43 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:43.288332 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-3076109d-3672-439d-b701-d9b87a044438 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:08:43.289301 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-3076109d-3672-439d-b701-d9b87a044438 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:08:43.297626 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a868fe4-e0bb-42e3-ba3b-88872e1a72c0 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:43.382243 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 429/854] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:08:43 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:08:43.421169 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a868fe4-e0bb-42e3-ba3b-88872e1a72c0 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:43.450324 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:43.456359 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fccfbd-bd15-42de-8b35-3805d727e797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:43.456969 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fccfbd-bd15-42de-8b35-3805d727e797 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:43.457251 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fccfbd-bd15-42de-8b35-3805d727e797 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:43.457518 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fccfbd-bd15-42de-8b35-3805d727e797 None admin] PATH_INFO: `/v3/services` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:43.458205 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fccfbd-bd15-42de-8b35-3805d727e797 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:43.458502 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fccfbd-bd15-42de-8b35-3805d727e797 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:43.459939 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fccfbd-bd15-42de-8b35-3805d727e797 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:43.471083 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 426/855] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Aug 24 17:08:43 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:43.530791 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3076109d-3672-439d-b701-d9b87a044438 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:43.606778 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:43.648439 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3076109d-3672-439d-b701-d9b87a044438 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:43.658252 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fccfbd-bd15-42de-8b35-3805d727e797 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:43.687591 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec9938d-17b0-4989-b2ef-35823cd695b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:43.688230 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec9938d-17b0-4989-b2ef-35823cd695b1 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:43.688514 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec9938d-17b0-4989-b2ef-35823cd695b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:43.688789 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec9938d-17b0-4989-b2ef-35823cd695b1 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:43.689377 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec9938d-17b0-4989-b2ef-35823cd695b1 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:43.690804 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec9938d-17b0-4989-b2ef-35823cd695b1 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:43.700630 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-6ec9938d-17b0-4989-b2ef-35823cd695b1 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 17:08:43.700630 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:43.700630 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 17:08:43.700630 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 17:08:43.700630 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 17:08:43.700630 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 17:08:43.700630 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 17:08:43.700630 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:08:43.700630 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 17:08:43.700630 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:08:43.700630 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 17:08:43.700630 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:08:43.700630 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:43.700630 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:43.700630 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:43.700630 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:43.701565 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:43.701565 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:43.701565 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:43.701565 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:43.701565 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:43.701565 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:43.701565 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:43.701565 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 17:08:43.701565 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 17:08:43.701565 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 17:08:43.701565 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 17:08:43.701565 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:43.701565 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:43.701565 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:43.701565 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:43.701565 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:43.701565 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:43.702452 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:43.702452 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:43.702452 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:43.702452 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:43.702452 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:43.702452 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:43.702452 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:43.702452 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:43.702452 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:43.702452 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:43.702452 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 17:08:43.702452 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:08:43.702452 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 17:08:43.702452 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:08:43.702452 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 430/856] 10.222.0.18 () {56 vars in 1132 bytes} [Wed Aug 24 17:08:43 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:43.707691 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec9938d-17b0-4989-b2ef-35823cd695b1 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:43.793139 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fccfbd-bd15-42de-8b35-3805d727e797 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:43.819414 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec9938d-17b0-4989-b2ef-35823cd695b1 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:43.847360 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:43.849165 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865af7a1-ff6d-4599-9d7c-f310c30f5230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:43.850291 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865af7a1-ff6d-4599-9d7c-f310c30f5230 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:43.851091 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865af7a1-ff6d-4599-9d7c-f310c30f5230 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:43.851834 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865af7a1-ff6d-4599-9d7c-f310c30f5230 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:43.858711 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9772ca3-5ec9-4343-90ee-b5e18286ed83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:43.859360 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9772ca3-5ec9-4343-90ee-b5e18286ed83 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:43.863075 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9772ca3-5ec9-4343-90ee-b5e18286ed83 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:43.863075 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9772ca3-5ec9-4343-90ee-b5e18286ed83 None admin] PATH_INFO: `/v3/services` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:43.863075 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9772ca3-5ec9-4343-90ee-b5e18286ed83 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:43.863075 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9772ca3-5ec9-4343-90ee-b5e18286ed83 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:43.863075 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9772ca3-5ec9-4343-90ee-b5e18286ed83 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:43.866310 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865af7a1-ff6d-4599-9d7c-f310c30f5230 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:08:43.867584 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 431/857] 10.222.0.18 () {56 vars in 1126 bytes} [Wed Aug 24 17:08:43 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:08:43.872115 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-865af7a1-ff6d-4599-9d7c-f310c30f5230 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:08:43.872115 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:43.872115 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:43.872115 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:43.872115 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:43.872115 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:43.872115 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:43.872115 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:43.872115 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:43.872115 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:43.872115 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:43.872115 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:43.872115 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:08:43.872115 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:08:43.872115 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:08:43.872115 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:08:43.873025 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:08:43.873025 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:08:43.873025 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:08:43.873025 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:08:43.873025 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:43.873025 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:43.873025 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:43.873025 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:43.873025 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:43.873025 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:43.873025 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:43.873025 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:43.873025 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:43.873025 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:43.873025 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:43.873025 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:43.873884 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:43.873884 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:43.873884 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:43.873884 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:43.873884 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:08:43.873884 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:08:43.873884 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:08:43.873884 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:08:43.873884 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:08:43.873884 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:08:43.873884 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:08:43.873884 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:08:43.874515 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9772ca3-5ec9-4343-90ee-b5e18286ed83 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:43.874586 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 427/858] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Aug 24 17:08:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:43.882736 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865af7a1-ff6d-4599-9d7c-f310c30f5230 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:43.951016 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:44.010555 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865af7a1-ff6d-4599-9d7c-f310c30f5230 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:44.012184 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9772ca3-5ec9-4343-90ee-b5e18286ed83 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:44.053880 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da22bccb-48ad-4eda-bab4-06e9ebfde8a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:44.054557 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da22bccb-48ad-4eda-bab4-06e9ebfde8a3 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:44.054766 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da22bccb-48ad-4eda-bab4-06e9ebfde8a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:44.055096 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da22bccb-48ad-4eda-bab4-06e9ebfde8a3 None admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:44.055750 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da22bccb-48ad-4eda-bab4-06e9ebfde8a3 None admin] RBAC: Adding query filter params (name=service) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:44.056054 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da22bccb-48ad-4eda-bab4-06e9ebfde8a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:44.056390 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36743b6-0aed-4c89-8348-6e2f42d8894f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:44.057238 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36743b6-0aed-4c89-8348-6e2f42d8894f None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:44.057428 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da22bccb-48ad-4eda-bab4-06e9ebfde8a3 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:44.057887 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36743b6-0aed-4c89-8348-6e2f42d8894f None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:44.058968 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36743b6-0aed-4c89-8348-6e2f42d8894f None admin] PATH_INFO: `/v3/services` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:44.060006 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36743b6-0aed-4c89-8348-6e2f42d8894f None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:44.060810 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36743b6-0aed-4c89-8348-6e2f42d8894f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:44.062344 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36743b6-0aed-4c89-8348-6e2f42d8894f None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:44.070933 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 428/859] 10.222.0.18 () {56 vars in 1103 bytes} [Wed Aug 24 17:08:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:08:44.078933 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 432/860] 10.222.0.18 () {56 vars in 1126 bytes} [Wed Aug 24 17:08:43 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:44.079139 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da22bccb-48ad-4eda-bab4-06e9ebfde8a3 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:44.186698 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da22bccb-48ad-4eda-bab4-06e9ebfde8a3 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:44.216004 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:44.223894 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364eed88-7f5f-4b52-979c-88fbb92a7e8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:44.224526 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364eed88-7f5f-4b52-979c-88fbb92a7e8e None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:44.224842 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364eed88-7f5f-4b52-979c-88fbb92a7e8e None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:44.225082 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364eed88-7f5f-4b52-979c-88fbb92a7e8e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:44.236788 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364eed88-7f5f-4b52-979c-88fbb92a7e8e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:44.238879 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364eed88-7f5f-4b52-979c-88fbb92a7e8e None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:44.253670 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-364eed88-7f5f-4b52-979c-88fbb92a7e8e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:08:44.253670 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:44.253670 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:44.253670 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:44.253670 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:44.253670 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:44.253670 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:44.253670 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:44.253670 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:44.253670 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:44.253670 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:44.253670 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:44.253670 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:08:44.253670 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:08:44.253670 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:08:44.253670 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:08:44.255117 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:44.255117 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:44.255117 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:44.255117 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:44.255117 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:44.255117 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:44.255117 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:44.255117 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:44.255117 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:44.255117 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:44.255117 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:44.255117 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:44.255117 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:44.255117 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:44.255117 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:44.255117 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:44.255117 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:08:44.256174 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:08:44.256174 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:08:44.256174 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:08:44.260706 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 429/861] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 24 17:08:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:08:44.263923 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36743b6-0aed-4c89-8348-6e2f42d8894f None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:44.321302 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:44.370840 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36743b6-0aed-4c89-8348-6e2f42d8894f None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:44.409911 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1945788b-31fc-41ed-9adf-7e8c9db816f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:44.410528 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1945788b-31fc-41ed-9adf-7e8c9db816f3 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:44.410812 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1945788b-31fc-41ed-9adf-7e8c9db816f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:44.411121 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1945788b-31fc-41ed-9adf-7e8c9db816f3 None admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:44.411769 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1945788b-31fc-41ed-9adf-7e8c9db816f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:44.412076 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1945788b-31fc-41ed-9adf-7e8c9db816f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:44.413447 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1945788b-31fc-41ed-9adf-7e8c9db816f3 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:44.426654 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 433/862] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 24 17:08:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:08:44.432066 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364eed88-7f5f-4b52-979c-88fbb92a7e8e None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:44.546014 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364eed88-7f5f-4b52-979c-88fbb92a7e8e None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:44.582097 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf36162-7139-4998-90c9-90c4e26ee0e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:44.582741 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf36162-7139-4998-90c9-90c4e26ee0e3 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:44.583061 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf36162-7139-4998-90c9-90c4e26ee0e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:44.583338 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf36162-7139-4998-90c9-90c4e26ee0e3 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:44.593832 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf36162-7139-4998-90c9-90c4e26ee0e3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:44.595365 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf36162-7139-4998-90c9-90c4e26ee0e3 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:44.606854 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-8cf36162-7139-4998-90c9-90c4e26ee0e3 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 17:08:44.606854 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:44.606854 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:44.606854 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:44.606854 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:44.606854 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:44.606854 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:44.606854 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:44.606854 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:44.606854 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:44.606854 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:44.606854 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:44.606854 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:08:44.606854 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:08:44.606854 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:08:44.606854 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:08:44.607831 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:44.607831 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:44.607831 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:08:44.607831 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:08:44.607831 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:08:44.607831 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:08:44.607831 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:44.607831 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:44.607831 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:44.607831 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:44.607831 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:44.607831 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:44.607831 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:44.607831 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:44.607831 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:44.607831 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:44.607831 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:44.607831 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:44.608756 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:44.608756 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:44.608756 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:08:44.608756 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:08:44.608756 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:08:44.608756 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:08:44.608756 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:08:44.608756 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:08:44.608756 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 17:08:44.608756 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:08:44.608756 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 430/863] 10.222.0.18 () {56 vars in 1117 bytes} [Wed Aug 24 17:08:44 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:44.616075 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1945788b-31fc-41ed-9adf-7e8c9db816f3 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:44.724015 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1945788b-31fc-41ed-9adf-7e8c9db816f3 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:44.738500 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:44.765354 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c4a45d-3164-44e9-84c1-9c00b0a024cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:44.765976 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c4a45d-3164-44e9-84c1-9c00b0a024cd None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:44.766261 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c4a45d-3164-44e9-84c1-9c00b0a024cd None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:44.766540 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c4a45d-3164-44e9-84c1-9c00b0a024cd None admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:44.767311 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c4a45d-3164-44e9-84c1-9c00b0a024cd None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:44.767621 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c4a45d-3164-44e9-84c1-9c00b0a024cd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:44.770374 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c4a45d-3164-44e9-84c1-9c00b0a024cd None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:44.790551 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 434/864] 10.222.0.18 () {56 vars in 1149 bytes} [Wed Aug 24 17:08:44 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:44.800023 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf36162-7139-4998-90c9-90c4e26ee0e3 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:44.831491 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:44.904386 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf36162-7139-4998-90c9-90c4e26ee0e3 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:44.938664 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802b612d-f6b8-42ea-9e18-fceb1066ffdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:44.939304 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802b612d-f6b8-42ea-9e18-fceb1066ffdb None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:44.939585 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802b612d-f6b8-42ea-9e18-fceb1066ffdb None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:44.939857 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802b612d-f6b8-42ea-9e18-fceb1066ffdb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:45.008842 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802b612d-f6b8-42ea-9e18-fceb1066ffdb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:45.010302 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802b612d-f6b8-42ea-9e18-fceb1066ffdb None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:45.045229 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:45.115090 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 431/865] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 24 17:08:44 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 17:08:45.121578 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c4a45d-3164-44e9-84c1-9c00b0a024cd None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:45.151650 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e7a44c-d761-41e7-a7f8-9ab42cbfb753 None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:45.151980 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e7a44c-d761-41e7-a7f8-9ab42cbfb753 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:45.152237 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e7a44c-d761-41e7-a7f8-9ab42cbfb753 None None] PATH_INFO: `/` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:45.154576 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 432/866] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:08:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 17:08:45.160126 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0451bc5c-87aa-4d75-95a8-6f4a11a7f3ce None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:45.160543 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0451bc5c-87aa-4d75-95a8-6f4a11a7f3ce None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:45.160821 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0451bc5c-87aa-4d75-95a8-6f4a11a7f3ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:45.182966 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-0451bc5c-87aa-4d75-95a8-6f4a11a7f3ce None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:08:45.185171 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-0451bc5c-87aa-4d75-95a8-6f4a11a7f3ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:08:45.239324 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c4a45d-3164-44e9-84c1-9c00b0a024cd None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:45.276844 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 433/867] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:08:45 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:08:45.284099 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a283c316-a3f6-4951-a508-201c54dd0cfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:45.284746 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a283c316-a3f6-4951-a508-201c54dd0cfa None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:45.285026 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a283c316-a3f6-4951-a508-201c54dd0cfa None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:45.285303 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a283c316-a3f6-4951-a508-201c54dd0cfa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:45.296616 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a283c316-a3f6-4951-a508-201c54dd0cfa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:45.298212 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a283c316-a3f6-4951-a508-201c54dd0cfa None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:45.313477 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-a283c316-a3f6-4951-a508-201c54dd0cfa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:08:45.313477 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:45.313477 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:45.313477 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:45.313477 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:45.313477 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:45.313477 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:45.313477 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:45.313477 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:45.313477 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:45.313477 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:45.313477 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:45.313477 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:08:45.313477 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:08:45.313477 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:08:45.313477 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:08:45.315547 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:45.315547 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:45.315547 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:45.315547 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:45.315547 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:45.315547 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:45.315547 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:45.315547 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:45.315547 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:45.315547 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:45.315547 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:45.315547 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:45.315547 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:45.315547 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:45.315547 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:45.315547 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:45.315547 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:08:45.317627 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:08:45.317627 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:08:45.317627 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:08:45.317627 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 435/868] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 24 17:08:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:08:45.324120 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0451bc5c-87aa-4d75-95a8-6f4a11a7f3ce None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:45.385692 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:45.427851 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0451bc5c-87aa-4d75-95a8-6f4a11a7f3ce None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:45.468178 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07059bb-be26-4ed4-a4a8-a792b0512194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:45.468788 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07059bb-be26-4ed4-a4a8-a792b0512194 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:45.469069 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07059bb-be26-4ed4-a4a8-a792b0512194 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:45.469345 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07059bb-be26-4ed4-a4a8-a792b0512194 None admin] PATH_INFO: `/v3/domains` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:45.469991 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07059bb-be26-4ed4-a4a8-a792b0512194 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:45.470294 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07059bb-be26-4ed4-a4a8-a792b0512194 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:45.471683 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07059bb-be26-4ed4-a4a8-a792b0512194 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:45.483982 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 434/869] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 24 17:08:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:45.489746 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a283c316-a3f6-4951-a508-201c54dd0cfa None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:45.516721 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:45.548925 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7cfc28-5301-47ae-9716-01eb8dd5ce6c None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:45.548925 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7cfc28-5301-47ae-9716-01eb8dd5ce6c None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:45.548925 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7cfc28-5301-47ae-9716-01eb8dd5ce6c None None] PATH_INFO: `/` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:45.550491 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 435/870] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:08:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 17:08:45.556765 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d11100c-5446-4000-84b4-5fd8fdf8cd61 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:45.557044 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d11100c-5446-4000-84b4-5fd8fdf8cd61 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:45.557326 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d11100c-5446-4000-84b4-5fd8fdf8cd61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:45.579382 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-6d11100c-5446-4000-84b4-5fd8fdf8cd61 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:08:45.581222 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-6d11100c-5446-4000-84b4-5fd8fdf8cd61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:08:45.608596 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a283c316-a3f6-4951-a508-201c54dd0cfa None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:45.646082 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c369a03d-29ca-4bc1-99bf-62f752aa0ceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:45.646631 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c369a03d-29ca-4bc1-99bf-62f752aa0ceb None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:45.647006 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c369a03d-29ca-4bc1-99bf-62f752aa0ceb None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:45.647208 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c369a03d-29ca-4bc1-99bf-62f752aa0ceb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:45.648433 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 436/871] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:08:45 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:08:45.649060 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d11100c-5446-4000-84b4-5fd8fdf8cd61 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:45.659390 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c369a03d-29ca-4bc1-99bf-62f752aa0ceb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:45.661854 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c369a03d-29ca-4bc1-99bf-62f752aa0ceb None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:45.676464 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-c369a03d-29ca-4bc1-99bf-62f752aa0ceb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:08:45.676464 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:45.676464 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:45.676464 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:45.676464 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:45.676464 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:45.676464 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:45.676464 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:45.676464 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:45.676464 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:45.676464 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:45.676464 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:45.676464 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:08:45.676464 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:08:45.676464 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:08:45.677391 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:08:45.677391 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:45.677391 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:45.677391 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:45.677391 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:45.677391 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:45.677391 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:45.677391 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:45.677391 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:45.677391 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:45.677391 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:45.677391 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:45.677391 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:45.677391 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:45.677391 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:45.677391 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:45.677391 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:45.678294 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:08:45.678294 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:08:45.678294 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:08:45.678294 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:08:45.678294 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:08:45.678294 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:08:45.678294 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:08:45.678294 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:08:45.681633 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 436/872] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 24 17:08:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:45.683875 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c369a03d-29ca-4bc1-99bf-62f752aa0ceb None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:45.719252 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:45.779313 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d11100c-5446-4000-84b4-5fd8fdf8cd61 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:45.780415 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:45.798781 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c369a03d-29ca-4bc1-99bf-62f752aa0ceb None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:45.828390 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86742f9f-eb97-445e-8f58-499ba1320b6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:45.829061 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86742f9f-eb97-445e-8f58-499ba1320b6e None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:45.829340 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86742f9f-eb97-445e-8f58-499ba1320b6e None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:45.829612 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86742f9f-eb97-445e-8f58-499ba1320b6e None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:45.830461 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86742f9f-eb97-445e-8f58-499ba1320b6e None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:45.832108 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86742f9f-eb97-445e-8f58-499ba1320b6e None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:45.841336 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622facfb-d5b8-4c48-b1a5-2dccc19c4609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:45.842262 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622facfb-d5b8-4c48-b1a5-2dccc19c4609 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:45.842756 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622facfb-d5b8-4c48-b1a5-2dccc19c4609 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:45.843297 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622facfb-d5b8-4c48-b1a5-2dccc19c4609 None admin] PATH_INFO: `/v3/projects` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:45.843900 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-86742f9f-eb97-445e-8f58-499ba1320b6e None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 17:08:45.843900 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:45.843900 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 17:08:45.843900 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 17:08:45.843900 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 17:08:45.843900 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 17:08:45.843900 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 17:08:45.843900 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:08:45.843900 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 17:08:45.843900 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:08:45.843900 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 17:08:45.843900 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:08:45.843900 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:45.843900 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:45.843900 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:45.843900 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:45.843900 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:45.844954 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:45.844954 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:45.844954 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:45.844954 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:45.844954 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:45.844954 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:45.844954 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 17:08:45.844954 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 17:08:45.844954 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 17:08:45.844954 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 17:08:45.844954 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:45.844954 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:45.844954 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:45.844954 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:45.844954 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:45.844954 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:45.845850 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:45.845850 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:45.845850 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:45.845850 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:45.845850 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:45.845850 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:45.845850 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:45.845850 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:45.845850 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:45.845850 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:45.845850 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 17:08:45.845850 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:08:45.845850 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 17:08:45.845850 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:08:45.847679 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622facfb-d5b8-4c48-b1a5-2dccc19c4609 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:45.848194 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622facfb-d5b8-4c48-b1a5-2dccc19c4609 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:45.849900 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622facfb-d5b8-4c48-b1a5-2dccc19c4609 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:45.850119 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 437/873] 10.222.0.18 () {56 vars in 1126 bytes} [Wed Aug 24 17:08:45 2022] GET /identity/v3/services/object-store => generated 93 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:45.850869 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86742f9f-eb97-445e-8f58-499ba1320b6e None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:45.864520 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 437/874] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Aug 24 17:08:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:08:45.869867 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622facfb-d5b8-4c48-b1a5-2dccc19c4609 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:45.964751 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86742f9f-eb97-445e-8f58-499ba1320b6e None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:45.978565 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:45.991506 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622facfb-d5b8-4c48-b1a5-2dccc19c4609 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:46.002167 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea124164-cea8-4942-964d-f2defdba3d3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:46.003224 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea124164-cea8-4942-964d-f2defdba3d3b None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:46.003601 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea124164-cea8-4942-964d-f2defdba3d3b None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:46.003877 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea124164-cea8-4942-964d-f2defdba3d3b None admin] PATH_INFO: `/v3/services` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:46.004500 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea124164-cea8-4942-964d-f2defdba3d3b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:46.004802 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea124164-cea8-4942-964d-f2defdba3d3b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:46.006277 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea124164-cea8-4942-964d-f2defdba3d3b None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:46.016261 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 438/875] 10.222.0.18 () {56 vars in 1122 bytes} [Wed Aug 24 17:08:45 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:46.016951 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea124164-cea8-4942-964d-f2defdba3d3b None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:46.026765 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5faaa431-ed3b-46f1-8659-d372019e0a37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:46.027456 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faaa431-ed3b-46f1-8659-d372019e0a37 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:46.027742 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faaa431-ed3b-46f1-8659-d372019e0a37 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:46.028016 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faaa431-ed3b-46f1-8659-d372019e0a37 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:46.028749 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5faaa431-ed3b-46f1-8659-d372019e0a37 None admin] RBAC: Adding query filter params (role.id=e7db2a847f56438bac13d0ff21c96b73, scope.project.id=46ab158bc3644c888557781b0630b0b8, user.id=0ccc6723f7514e52863d102de3839951) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:46.029061 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5faaa431-ed3b-46f1-8659-d372019e0a37 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:46.030488 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5faaa431-ed3b-46f1-8659-d372019e0a37 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:46.046944 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 438/876] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 24 17:08:45 2022] GET /identity/v3/role_assignments?user.id=0ccc6723f7514e52863d102de3839951&scope.project.id=46ab158bc3644c888557781b0630b0b8&role.id=e7db2a847f56438bac13d0ff21c96b73 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:08:46.046944 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5faaa431-ed3b-46f1-8659-d372019e0a37 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:46.047216 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:46.143506 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea124164-cea8-4942-964d-f2defdba3d3b None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:46.161135 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5faaa431-ed3b-46f1-8659-d372019e0a37 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:46.183010 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507db8af-1ef6-43ca-baff-a00a37a3bc2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:46.184183 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507db8af-1ef6-43ca-baff-a00a37a3bc2a None admin] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:46.184464 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507db8af-1ef6-43ca-baff-a00a37a3bc2a None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:46.184735 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507db8af-1ef6-43ca-baff-a00a37a3bc2a None admin] PATH_INFO: `/v3/services` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:46.185388 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507db8af-1ef6-43ca-baff-a00a37a3bc2a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:46.186746 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507db8af-1ef6-43ca-baff-a00a37a3bc2a None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:46.201829 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34f7e25-9d8e-4d38-87b8-491d725e077e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:46.202441 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34f7e25-9d8e-4d38-87b8-491d725e077e None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:46.202719 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34f7e25-9d8e-4d38-87b8-491d725e077e None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:46.203018 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34f7e25-9d8e-4d38-87b8-491d725e077e None admin] PATH_INFO: `/v3/services` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:46.203657 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34f7e25-9d8e-4d38-87b8-491d725e077e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:46.203951 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34f7e25-9d8e-4d38-87b8-491d725e077e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:46.205312 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34f7e25-9d8e-4d38-87b8-491d725e077e None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:46.209957 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 439/877] 10.222.0.18 () {56 vars in 1122 bytes} [Wed Aug 24 17:08:46 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:46.211242 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a473954a-9b5d-4fe2-97ad-3f5d7647733f None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:46.211446 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a473954a-9b5d-4fe2-97ad-3f5d7647733f None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:46.211718 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a473954a-9b5d-4fe2-97ad-3f5d7647733f None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:46.212926 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 440/878] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:08:46 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:08:46.219056 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842077ab-1e26-4541-8d23-520d1ec46ffb None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:46.219238 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842077ab-1e26-4541-8d23-520d1ec46ffb None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:46.219520 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842077ab-1e26-4541-8d23-520d1ec46ffb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:46.240613 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-842077ab-1e26-4541-8d23-520d1ec46ffb None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:08:46.242085 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-842077ab-1e26-4541-8d23-520d1ec46ffb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:08:46.303585 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 439/879] 10.222.0.18 () {60 vars in 1141 bytes} [Wed Aug 24 17:08:46 2022] POST /identity/v3/services => generated 251 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:08:46.319320 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:46.332454 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 441/880] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:08:46 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 17:08:46.608131 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507db8af-1ef6-43ca-baff-a00a37a3bc2a None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:46.646798 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:46.713508 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507db8af-1ef6-43ca-baff-a00a37a3bc2a None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:46.749586 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6b1cdf-f57c-44e0-8c26-5256a5615bcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:46.750136 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6b1cdf-f57c-44e0-8c26-5256a5615bcb None admin] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:46.750416 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6b1cdf-f57c-44e0-8c26-5256a5615bcb None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:46.750692 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6b1cdf-f57c-44e0-8c26-5256a5615bcb None admin] PATH_INFO: `/v3/services` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:46.751642 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6b1cdf-f57c-44e0-8c26-5256a5615bcb None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:46.753069 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6b1cdf-f57c-44e0-8c26-5256a5615bcb None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:46.882420 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 440/881] 10.222.0.18 () {60 vars in 1141 bytes} [Wed Aug 24 17:08:46 2022] POST /identity/v3/services => generated 269 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:08:48.132373 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f650dbeb-a574-4df8-8291-8127f29bcf00 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:48.132908 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f650dbeb-a574-4df8-8291-8127f29bcf00 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:48.132908 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f650dbeb-a574-4df8-8291-8127f29bcf00 None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:48.134383 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 442/882] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:08:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:08:48.135728 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798c0992-c6ae-41ef-b938-27ccb4f98c76 None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:48.136113 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798c0992-c6ae-41ef-b938-27ccb4f98c76 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:48.136270 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798c0992-c6ae-41ef-b938-27ccb4f98c76 None None] PATH_INFO: `/` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:48.138419 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 441/883] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:08:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:08:48.142153 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1306cad-647a-4e87-8715-ef3ed0986984 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:48.142432 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1306cad-647a-4e87-8715-ef3ed0986984 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:48.142704 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1306cad-647a-4e87-8715-ef3ed0986984 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:48.146369 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f44fa7-c970-47d3-8fc4-7f587ff1b62e None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:48.146998 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f44fa7-c970-47d3-8fc4-7f587ff1b62e None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:48.146998 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f44fa7-c970-47d3-8fc4-7f587ff1b62e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:48.163649 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-c1306cad-647a-4e87-8715-ef3ed0986984 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:08:48.164978 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-c1306cad-647a-4e87-8715-ef3ed0986984 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:08:48.169051 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-53f44fa7-c970-47d3-8fc4-7f587ff1b62e None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:08:48.170380 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-53f44fa7-c970-47d3-8fc4-7f587ff1b62e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:08:48.247184 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 443/884] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:08:48 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 17:08:48.263043 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 442/885] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:08:48 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:08:48.266388 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8b4464-e2eb-4b6a-8d4c-f90e126469c3 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:48.270924 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8b4464-e2eb-4b6a-8d4c-f90e126469c3 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:48.270924 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8b4464-e2eb-4b6a-8d4c-f90e126469c3 None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:48.270924 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 444/886] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:08:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:08:48.276310 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f89ee2e-48de-4375-ae86-e4d4354a607c None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:48.276597 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f89ee2e-48de-4375-ae86-e4d4354a607c None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:48.276872 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f89ee2e-48de-4375-ae86-e4d4354a607c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:48.295724 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-6f89ee2e-48de-4375-ae86-e4d4354a607c None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:08:48.296899 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-6f89ee2e-48de-4375-ae86-e4d4354a607c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:08:48.317539 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:48.370985 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 443/887] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:08:48 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 17:08:48.514383 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8b4464-e2eb-4b6a-8d4c-f90e126469c3 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:48.542220 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f89ee2e-48de-4375-ae86-e4d4354a607c None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:48.622326 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8b4464-e2eb-4b6a-8d4c-f90e126469c3 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:48.649486 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:48.653750 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f89ee2e-48de-4375-ae86-e4d4354a607c None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:48.660968 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ee0560-4f1f-405b-8125-fae7195a0d76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:48.661616 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ee0560-4f1f-405b-8125-fae7195a0d76 None admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:48.661972 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ee0560-4f1f-405b-8125-fae7195a0d76 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:48.662247 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ee0560-4f1f-405b-8125-fae7195a0d76 None admin] PATH_INFO: `/v3/services` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:48.662957 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ee0560-4f1f-405b-8125-fae7195a0d76 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:48.664465 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ee0560-4f1f-405b-8125-fae7195a0d76 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:48.691057 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba18612b-2277-42df-9fd9-d21afe01efb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:48.691694 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba18612b-2277-42df-9fd9-d21afe01efb0 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:48.691971 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba18612b-2277-42df-9fd9-d21afe01efb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:48.692244 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba18612b-2277-42df-9fd9-d21afe01efb0 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:48.703155 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba18612b-2277-42df-9fd9-d21afe01efb0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:08:48.707778 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-ba18612b-2277-42df-9fd9-d21afe01efb0 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 17:08:48.707778 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:48.707778 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:48.707778 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:48.707778 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:48.707778 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:48.707778 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:48.707778 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:48.707778 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:48.707778 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:48.707778 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:48.707778 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:48.707778 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:08:48.707778 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:08:48.707778 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:08:48.707778 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:08:48.708995 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:08:48.708995 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:08:48.708995 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:08:48.708995 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:08:48.708995 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:48.708995 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:48.708995 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:48.708995 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:48.708995 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:48.708995 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:48.708995 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:48.708995 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:48.708995 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:48.708995 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:48.708995 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:48.708995 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:48.710189 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:48.710189 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:48.710189 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:48.710189 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:48.710189 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:08:48.710189 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:08:48.710189 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:08:48.710189 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:08:48.710189 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:08:48.710189 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:08:48.710189 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 17:08:48.710189 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:08:48.710189 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 444/888] 10.222.0.18 () {56 vars in 1120 bytes} [Wed Aug 24 17:08:48 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:48.710189 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba18612b-2277-42df-9fd9-d21afe01efb0 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:48.730525 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:48.738800 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 445/889] 10.222.0.18 () {60 vars in 1141 bytes} [Wed Aug 24 17:08:48 2022] POST /identity/v3/services => generated 241 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:08:48.739685 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ee0560-4f1f-405b-8125-fae7195a0d76 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:48.848017 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba18612b-2277-42df-9fd9-d21afe01efb0 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:48.878480 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ee0560-4f1f-405b-8125-fae7195a0d76 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:48.892402 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdacb2e4-a052-434a-8a08-ae6c4ab01f84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:48.893040 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdacb2e4-a052-434a-8a08-ae6c4ab01f84 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:48.893322 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdacb2e4-a052-434a-8a08-ae6c4ab01f84 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:48.893597 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdacb2e4-a052-434a-8a08-ae6c4ab01f84 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:48.894183 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdacb2e4-a052-434a-8a08-ae6c4ab01f84 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:48.895583 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdacb2e4-a052-434a-8a08-ae6c4ab01f84 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:48.904741 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-bdacb2e4-a052-434a-8a08-ae6c4ab01f84 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 17:08:48.904741 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:48.904741 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 17:08:48.904741 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 17:08:48.904741 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 17:08:48.904741 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 17:08:48.904741 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 17:08:48.904741 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:08:48.904741 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 17:08:48.904741 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:08:48.904741 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 17:08:48.904741 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:08:48.904741 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:48.904741 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:48.904741 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:48.904741 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:48.904741 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:48.905745 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:48.905745 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:48.905745 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:48.905745 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:48.905745 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:48.905745 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:48.905745 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 17:08:48.905745 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 17:08:48.905745 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 17:08:48.905745 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 17:08:48.905745 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:48.905745 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:48.905745 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:48.905745 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:48.905745 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:48.905745 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:48.907751 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:48.907751 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:48.907751 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:48.907751 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:48.907751 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:48.907751 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:48.907751 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:48.907751 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:48.907751 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:48.907751 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:48.907751 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 17:08:48.907751 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:08:48.907751 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 17:08:48.907751 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:08:48.907751 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 445/890] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Aug 24 17:08:48 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:48.908561 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e260da-ad72-482c-a583-fabab1465f80 None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:48.908561 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e260da-ad72-482c-a583-fabab1465f80 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:48.908561 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e260da-ad72-482c-a583-fabab1465f80 None None] PATH_INFO: `/` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:48.909100 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 446/891] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:08:48 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:08:48.913749 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e260da-ad72-482c-a583-fabab1465f80 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:48.918265 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a089e7e9-595a-41a1-9747-dfd240449ce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:48.918755 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a089e7e9-595a-41a1-9747-dfd240449ce4 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:48.919385 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a089e7e9-595a-41a1-9747-dfd240449ce4 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:48.919385 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a089e7e9-595a-41a1-9747-dfd240449ce4 None admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:48.919986 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a089e7e9-595a-41a1-9747-dfd240449ce4 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:48.920288 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a089e7e9-595a-41a1-9747-dfd240449ce4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:48.921661 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a089e7e9-595a-41a1-9747-dfd240449ce4 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:48.931322 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 446/892] 10.222.0.18 () {56 vars in 1115 bytes} [Wed Aug 24 17:08:48 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:48.932448 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaca276-7c97-4b9b-8abb-cccd00541bc0 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:48.932815 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaca276-7c97-4b9b-8abb-cccd00541bc0 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:48.933098 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaca276-7c97-4b9b-8abb-cccd00541bc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:48.951776 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-bfaca276-7c97-4b9b-8abb-cccd00541bc0 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:08:48.952962 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-bfaca276-7c97-4b9b-8abb-cccd00541bc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:08:48.988560 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:49.013339 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:49.034077 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 447/893] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:08:48 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:08:49.035766 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaca276-7c97-4b9b-8abb-cccd00541bc0 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:49.035968 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e260da-ad72-482c-a583-fabab1465f80 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:49.072967 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005fbb30-90af-47cc-85b3-8d741fcfad58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:49.076069 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005fbb30-90af-47cc-85b3-8d741fcfad58 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:49.076069 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005fbb30-90af-47cc-85b3-8d741fcfad58 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:49.076069 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005fbb30-90af-47cc-85b3-8d741fcfad58 None admin] PATH_INFO: `/v3/services` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:49.076069 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005fbb30-90af-47cc-85b3-8d741fcfad58 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:49.076069 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005fbb30-90af-47cc-85b3-8d741fcfad58 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:49.077310 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005fbb30-90af-47cc-85b3-8d741fcfad58 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:49.090410 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 447/894] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Aug 24 17:08:48 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:49.091310 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005fbb30-90af-47cc-85b3-8d741fcfad58 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:49.151644 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaca276-7c97-4b9b-8abb-cccd00541bc0 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:49.188858 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec1d6d3-0f21-4586-b93a-db7ea39627ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:49.189595 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec1d6d3-0f21-4586-b93a-db7ea39627ad None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:49.189883 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec1d6d3-0f21-4586-b93a-db7ea39627ad None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:49.190317 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec1d6d3-0f21-4586-b93a-db7ea39627ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:49.193192 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005fbb30-90af-47cc-85b3-8d741fcfad58 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:49.201560 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec1d6d3-0f21-4586-b93a-db7ea39627ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:49.203085 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec1d6d3-0f21-4586-b93a-db7ea39627ad None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:49.220030 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-cec1d6d3-0f21-4586-b93a-db7ea39627ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:08:49.220030 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:49.220030 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:49.220030 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:49.220030 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:49.220030 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:49.220030 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:49.220030 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:49.220030 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:49.220030 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:49.220030 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:49.220030 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:49.220030 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:08:49.220030 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:08:49.220030 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:08:49.220030 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:08:49.221010 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:49.221010 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:49.221010 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:49.221010 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:49.221010 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:49.221010 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:49.221010 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:49.221010 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:49.221010 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:49.221010 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:49.221010 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:49.221010 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:49.221010 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:49.221010 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:49.221010 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:49.221010 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:49.221010 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:08:49.221907 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:08:49.221907 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:08:49.221907 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:08:49.221907 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 448/895] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 24 17:08:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:49.226809 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec1d6d3-0f21-4586-b93a-db7ea39627ad None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:49.237071 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e966a7b-0e0f-4b3a-aba9-df32a555d8d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:49.237689 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e966a7b-0e0f-4b3a-aba9-df32a555d8d7 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:49.237973 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e966a7b-0e0f-4b3a-aba9-df32a555d8d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:49.238251 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e966a7b-0e0f-4b3a-aba9-df32a555d8d7 None admin] PATH_INFO: `/v3/services` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:49.239377 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e966a7b-0e0f-4b3a-aba9-df32a555d8d7 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:49.239694 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e966a7b-0e0f-4b3a-aba9-df32a555d8d7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:49.241064 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e966a7b-0e0f-4b3a-aba9-df32a555d8d7 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:49.251301 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 448/896] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Aug 24 17:08:49 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:49.254413 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e966a7b-0e0f-4b3a-aba9-df32a555d8d7 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:49.263673 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:49.268936 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:49.353671 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec1d6d3-0f21-4586-b93a-db7ea39627ad None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:49.373283 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e966a7b-0e0f-4b3a-aba9-df32a555d8d7 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:49.390871 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612245ef-c105-4de6-88f9-396ef47a118d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:49.391609 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612245ef-c105-4de6-88f9-396ef47a118d None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:49.391894 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612245ef-c105-4de6-88f9-396ef47a118d None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:49.392256 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612245ef-c105-4de6-88f9-396ef47a118d None admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:49.392900 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612245ef-c105-4de6-88f9-396ef47a118d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:49.393198 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612245ef-c105-4de6-88f9-396ef47a118d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:49.394572 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612245ef-c105-4de6-88f9-396ef47a118d None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:49.415841 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 449/897] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 24 17:08:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:49.416017 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77a6e6d-261c-4429-8725-54521ef10b35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:49.416132 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612245ef-c105-4de6-88f9-396ef47a118d None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:49.416221 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77a6e6d-261c-4429-8725-54521ef10b35 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:49.416221 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77a6e6d-261c-4429-8725-54521ef10b35 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:49.416221 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77a6e6d-261c-4429-8725-54521ef10b35 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:49.416221 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77a6e6d-261c-4429-8725-54521ef10b35 None admin] RBAC: Adding query filter params (interface=public, service_id=1cf69e0c7936496ebe69ea15bf64ae30, region_id=RegionOne) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:49.416221 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77a6e6d-261c-4429-8725-54521ef10b35 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:49.416221 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77a6e6d-261c-4429-8725-54521ef10b35 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:49.418832 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 449/898] 10.222.0.18 () {56 vars in 1251 bytes} [Wed Aug 24 17:08:49 2022] GET /identity/v3/endpoints?service_id=1cf69e0c7936496ebe69ea15bf64ae30&interface=public®ion_id=RegionOne => generated 194 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:49.419668 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77a6e6d-261c-4429-8725-54521ef10b35 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:49.494726 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:49.508202 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:49.524513 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77a6e6d-261c-4429-8725-54521ef10b35 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:49.529358 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612245ef-c105-4de6-88f9-396ef47a118d None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:49.566576 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c7fb0a-452f-44ae-b992-088052e00e72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:49.566816 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede61c92-6b86-4edd-80af-c56d2088f61b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:49.567379 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c7fb0a-452f-44ae-b992-088052e00e72 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:49.567479 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede61c92-6b86-4edd-80af-c56d2088f61b None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:49.567735 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede61c92-6b86-4edd-80af-c56d2088f61b None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:49.567847 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c7fb0a-452f-44ae-b992-088052e00e72 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:49.568004 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede61c92-6b86-4edd-80af-c56d2088f61b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:49.568096 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c7fb0a-452f-44ae-b992-088052e00e72 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:49.568732 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c7fb0a-452f-44ae-b992-088052e00e72 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:49.570168 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c7fb0a-452f-44ae-b992-088052e00e72 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:49.576311 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede61c92-6b86-4edd-80af-c56d2088f61b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:49.577895 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede61c92-6b86-4edd-80af-c56d2088f61b None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:49.579688 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-68c7fb0a-452f-44ae-b992-088052e00e72 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 17:08:49.579688 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:49.579688 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 17:08:49.579688 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 17:08:49.579688 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 17:08:49.579688 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 17:08:49.579688 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 17:08:49.579688 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:08:49.579688 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 17:08:49.579688 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:08:49.579688 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 17:08:49.579688 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:08:49.579688 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:49.579688 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:49.579688 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:49.579688 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:49.580765 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:49.580765 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:49.580765 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:49.580765 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:49.580765 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:49.580765 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:49.580765 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:49.580765 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 17:08:49.580765 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 17:08:49.580765 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 17:08:49.580765 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 17:08:49.580765 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:49.580765 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:49.580765 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:49.580765 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:49.580765 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:49.580765 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:49.581636 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:49.581636 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:49.581636 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:49.581636 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:49.581636 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:49.581636 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:49.581636 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:49.581636 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:49.581636 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:49.581636 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:49.581636 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 17:08:49.581636 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:08:49.581636 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 17:08:49.581636 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:08:49.581636 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 450/899] 10.222.0.18 () {56 vars in 1132 bytes} [Wed Aug 24 17:08:49 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:49.582681 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c7fb0a-452f-44ae-b992-088052e00e72 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:49.595691 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-ede61c92-6b86-4edd-80af-c56d2088f61b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 17:08:49.595691 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:49.595691 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:49.595691 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:49.595691 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:49.595691 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:49.595691 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:49.595691 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:49.595691 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:49.595691 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:49.595691 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:49.595691 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:49.595691 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:08:49.595691 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:08:49.595691 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:08:49.595691 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:08:49.596890 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:49.596890 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:49.596890 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:08:49.596890 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:08:49.596890 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:08:49.596890 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:08:49.596890 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:49.596890 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:49.596890 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:49.596890 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:49.596890 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:49.596890 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:49.596890 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:49.596890 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:49.596890 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:49.596890 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:49.596890 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:49.596890 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:49.598109 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:49.598109 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:49.598109 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:08:49.598109 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:08:49.598109 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:08:49.598109 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:08:49.598109 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:08:49.598109 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:08:49.598109 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 17:08:49.598109 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:08:49.601449 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 450/900] 10.222.0.18 () {56 vars in 1117 bytes} [Wed Aug 24 17:08:49 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:49.602279 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede61c92-6b86-4edd-80af-c56d2088f61b None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:49.695687 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c7fb0a-452f-44ae-b992-088052e00e72 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:49.712356 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede61c92-6b86-4edd-80af-c56d2088f61b None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:49.736862 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec957cef-3c61-4434-a18d-f8d3ea34aa9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:49.737476 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec957cef-3c61-4434-a18d-f8d3ea34aa9c None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:49.737756 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec957cef-3c61-4434-a18d-f8d3ea34aa9c None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:49.738024 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec957cef-3c61-4434-a18d-f8d3ea34aa9c None admin] PATH_INFO: `/v3/services` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:49.738622 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec957cef-3c61-4434-a18d-f8d3ea34aa9c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:49.740019 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec957cef-3c61-4434-a18d-f8d3ea34aa9c None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:49.743916 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:49.754932 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 451/901] 10.222.0.18 () {56 vars in 1087 bytes} [Wed Aug 24 17:08:49 2022] GET /identity/v3/services => generated 1497 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:08:49.754932 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec957cef-3c61-4434-a18d-f8d3ea34aa9c None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:49.758754 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e9cb33-7fce-47c9-9818-de4ee91ac662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:49.759909 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e9cb33-7fce-47c9-9818-de4ee91ac662 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:49.759909 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e9cb33-7fce-47c9-9818-de4ee91ac662 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:49.760306 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e9cb33-7fce-47c9-9818-de4ee91ac662 None admin] PATH_INFO: `/v3/services` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:49.761098 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e9cb33-7fce-47c9-9818-de4ee91ac662 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:49.761515 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e9cb33-7fce-47c9-9818-de4ee91ac662 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:49.763045 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e9cb33-7fce-47c9-9818-de4ee91ac662 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:49.773760 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:49.774720 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 451/902] 10.222.0.18 () {56 vars in 1126 bytes} [Wed Aug 24 17:08:49 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:49.775782 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e9cb33-7fce-47c9-9818-de4ee91ac662 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:49.902223 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e9cb33-7fce-47c9-9818-de4ee91ac662 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:49.908239 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec957cef-3c61-4434-a18d-f8d3ea34aa9c None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:49.947815 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04643491-9d4e-479f-96d6-b85906349f3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:49.948469 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04643491-9d4e-479f-96d6-b85906349f3c None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:49.948815 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04643491-9d4e-479f-96d6-b85906349f3c None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:49.949125 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04643491-9d4e-479f-96d6-b85906349f3c None admin] PATH_INFO: `/v3/services` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:49.949753 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04643491-9d4e-479f-96d6-b85906349f3c None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:49.950150 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04643491-9d4e-479f-96d6-b85906349f3c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:49.951639 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04643491-9d4e-479f-96d6-b85906349f3c None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:49.952145 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadcc8cb-ff5f-4dde-a79e-17f556a48112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:49.952747 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadcc8cb-ff5f-4dde-a79e-17f556a48112 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:49.953091 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadcc8cb-ff5f-4dde-a79e-17f556a48112 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:49.953279 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadcc8cb-ff5f-4dde-a79e-17f556a48112 None admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:49.954003 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadcc8cb-ff5f-4dde-a79e-17f556a48112 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:49.954311 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadcc8cb-ff5f-4dde-a79e-17f556a48112 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:49.955811 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadcc8cb-ff5f-4dde-a79e-17f556a48112 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:49.957683 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 452/903] 10.222.0.18 () {56 vars in 1126 bytes} [Wed Aug 24 17:08:49 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:08:49.963842 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04643491-9d4e-479f-96d6-b85906349f3c None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:49.978942 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 452/904] 10.222.0.18 () {56 vars in 1149 bytes} [Wed Aug 24 17:08:49 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:49.983249 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadcc8cb-ff5f-4dde-a79e-17f556a48112 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:50.047622 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:50.067758 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:50.097642 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04643491-9d4e-479f-96d6-b85906349f3c None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:50.104023 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadcc8cb-ff5f-4dde-a79e-17f556a48112 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:50.133862 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f5bfa4-6c7b-4b68-984c-f8ca89e9b2a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:50.134573 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f5bfa4-6c7b-4b68-984c-f8ca89e9b2a0 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:50.134930 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f5bfa4-6c7b-4b68-984c-f8ca89e9b2a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:50.135281 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f5bfa4-6c7b-4b68-984c-f8ca89e9b2a0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:50.136079 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f5bfa4-6c7b-4b68-984c-f8ca89e9b2a0 None admin] RBAC: Adding query filter params (interface=public, service_id=84ec07f9b9384ccf820f914768145fa5, region_id=RegionOne) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:50.136496 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f5bfa4-6c7b-4b68-984c-f8ca89e9b2a0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:50.138136 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f5bfa4-6c7b-4b68-984c-f8ca89e9b2a0 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:50.142702 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8439454-7fd6-4106-83b6-21b5df66213b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:50.143550 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8439454-7fd6-4106-83b6-21b5df66213b None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:50.143996 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 453/905] 10.222.0.18 () {56 vars in 1251 bytes} [Wed Aug 24 17:08:49 2022] GET /identity/v3/endpoints?service_id=84ec07f9b9384ccf820f914768145fa5&interface=public®ion_id=RegionOne => generated 194 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:08:50.144212 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8439454-7fd6-4106-83b6-21b5df66213b None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:50.144489 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8439454-7fd6-4106-83b6-21b5df66213b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:50.152608 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f5bfa4-6c7b-4b68-984c-f8ca89e9b2a0 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:50.213818 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8439454-7fd6-4106-83b6-21b5df66213b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:50.215607 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8439454-7fd6-4106-83b6-21b5df66213b None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:50.260397 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f5bfa4-6c7b-4b68-984c-f8ca89e9b2a0 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:50.286713 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:50.287049 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:50.297890 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d577118-50a8-4048-9cdf-81c60780d842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:50.298503 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d577118-50a8-4048-9cdf-81c60780d842 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:50.298782 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d577118-50a8-4048-9cdf-81c60780d842 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:50.304904 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d577118-50a8-4048-9cdf-81c60780d842 None admin] PATH_INFO: `/v3/services` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:50.304904 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d577118-50a8-4048-9cdf-81c60780d842 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:50.304904 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d577118-50a8-4048-9cdf-81c60780d842 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:50.315303 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 454/906] 10.222.0.18 () {56 vars in 1087 bytes} [Wed Aug 24 17:08:50 2022] GET /identity/v3/services => generated 1497 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:08:50.325330 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 453/907] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 24 17:08:49 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 17:08:50.330787 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d577118-50a8-4048-9cdf-81c60780d842 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:50.440909 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d577118-50a8-4048-9cdf-81c60780d842 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:50.482093 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532a8615-f505-475c-aeff-9b3279075eab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:50.483101 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532a8615-f505-475c-aeff-9b3279075eab None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:50.483591 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532a8615-f505-475c-aeff-9b3279075eab None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:50.484085 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532a8615-f505-475c-aeff-9b3279075eab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:50.499613 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532a8615-f505-475c-aeff-9b3279075eab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:50.501377 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532a8615-f505-475c-aeff-9b3279075eab None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:50.516384 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-532a8615-f505-475c-aeff-9b3279075eab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:08:50.516384 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:50.516384 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:50.516384 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:50.516384 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:50.516384 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:50.516384 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:50.516384 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:50.516384 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:50.516384 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:50.516384 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:50.516384 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:50.516384 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:08:50.516384 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:08:50.516384 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:08:50.516384 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:08:50.517395 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:50.517395 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:50.517395 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:50.517395 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:50.517395 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:50.517395 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:50.517395 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:50.517395 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:50.517395 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:50.517395 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:50.517395 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:50.517395 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:50.517395 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:50.517395 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:50.517395 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:50.517395 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:50.517395 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:08:50.518292 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:08:50.518292 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:08:50.518292 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:08:50.518292 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 455/908] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 24 17:08:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:50.524793 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8439454-7fd6-4106-83b6-21b5df66213b None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:50.645244 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8439454-7fd6-4106-83b6-21b5df66213b None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:50.681048 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f19c81e-137e-4c5b-bb16-c0a985d5ede8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:50.682005 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f19c81e-137e-4c5b-bb16-c0a985d5ede8 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:50.682480 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f19c81e-137e-4c5b-bb16-c0a985d5ede8 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:50.683016 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f19c81e-137e-4c5b-bb16-c0a985d5ede8 None admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:50.684648 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f19c81e-137e-4c5b-bb16-c0a985d5ede8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:50.685171 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f19c81e-137e-4c5b-bb16-c0a985d5ede8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:50.686726 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f19c81e-137e-4c5b-bb16-c0a985d5ede8 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:50.698193 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e70ea7-505e-459a-bd22-4de251a2c059 None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:50.698464 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e70ea7-505e-459a-bd22-4de251a2c059 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:50.698742 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e70ea7-505e-459a-bd22-4de251a2c059 None None] PATH_INFO: `/` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:50.700043 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 454/909] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 24 17:08:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:50.700318 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 456/910] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:08:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:08:50.705541 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e70ea7-505e-459a-bd22-4de251a2c059 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:50.705685 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a8612e-725a-4a9a-9142-cd95efa59391 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:50.705873 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a8612e-725a-4a9a-9142-cd95efa59391 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:50.706084 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a8612e-725a-4a9a-9142-cd95efa59391 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:50.723932 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-c7a8612e-725a-4a9a-9142-cd95efa59391 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:08:50.725115 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-c7a8612e-725a-4a9a-9142-cd95efa59391 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:08:50.751785 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:50.756970 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:50.805492 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 455/911] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:08:50 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 17:08:50.829212 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e70ea7-505e-459a-bd22-4de251a2c059 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:50.864965 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2777f3-fa23-4a23-8559-5cba84fb79af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:50.865960 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2777f3-fa23-4a23-8559-5cba84fb79af None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:50.866457 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2777f3-fa23-4a23-8559-5cba84fb79af None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:50.866952 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2777f3-fa23-4a23-8559-5cba84fb79af None admin] PATH_INFO: `/v3/projects/service` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:50.873888 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2777f3-fa23-4a23-8559-5cba84fb79af None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:50.875562 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2777f3-fa23-4a23-8559-5cba84fb79af None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:50.887241 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-6a2777f3-fa23-4a23-8559-5cba84fb79af None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:08:50.887241 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:50.887241 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:50.887241 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:50.887241 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:50.887241 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:50.887241 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:50.887241 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:50.887241 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:50.887241 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:50.887241 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:50.887241 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:50.887241 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:08:50.887241 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:08:50.887241 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:08:50.888151 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:08:50.888151 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:50.888151 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:50.888151 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:50.888151 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:50.888151 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:50.888151 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:50.888151 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:50.888151 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:50.888151 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:50.888151 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:50.888151 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:50.888151 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:50.888151 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:50.888151 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:50.888151 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:50.888151 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:50.889128 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:08:50.889128 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:08:50.889128 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:08:50.889128 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:08:50.889128 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:08:50.889128 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:08:50.889128 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:08:50.889128 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:08:50.891483 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 457/912] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 24 17:08:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:50.897342 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a8612e-725a-4a9a-9142-cd95efa59391 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:51.002334 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a8612e-725a-4a9a-9142-cd95efa59391 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:51.037886 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdc287f-e824-42a0-89be-7bfd2fedfd64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:51.038817 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdc287f-e824-42a0-89be-7bfd2fedfd64 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:51.039332 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdc287f-e824-42a0-89be-7bfd2fedfd64 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:51.039805 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdc287f-e824-42a0-89be-7bfd2fedfd64 None admin] PATH_INFO: `/v3/projects` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:51.040706 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdc287f-e824-42a0-89be-7bfd2fedfd64 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:51.041229 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdc287f-e824-42a0-89be-7bfd2fedfd64 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:51.043458 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdc287f-e824-42a0-89be-7bfd2fedfd64 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:51.056121 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 456/913] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Aug 24 17:08:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:51.063186 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2777f3-fa23-4a23-8559-5cba84fb79af None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:51.083310 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdc287f-e824-42a0-89be-7bfd2fedfd64 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:51.104287 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:51.173991 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:51.177243 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2777f3-fa23-4a23-8559-5cba84fb79af None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:51.191908 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdc287f-e824-42a0-89be-7bfd2fedfd64 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:51.212130 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3e475f-96ab-4c94-9c26-4a8907b7fe0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:51.213107 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3e475f-96ab-4c94-9c26-4a8907b7fe0c None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:51.213613 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3e475f-96ab-4c94-9c26-4a8907b7fe0c None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:51.214083 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3e475f-96ab-4c94-9c26-4a8907b7fe0c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:51.215605 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3e475f-96ab-4c94-9c26-4a8907b7fe0c None admin] RBAC: Adding query filter params (role.id=967c5aaa9ba746a7acaa59f3e9691b31, scope.project.id=46ab158bc3644c888557781b0630b0b8, user.id=0ccc6723f7514e52863d102de3839951) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:51.216116 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3e475f-96ab-4c94-9c26-4a8907b7fe0c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:51.217721 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3e475f-96ab-4c94-9c26-4a8907b7fe0c None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:51.230569 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470de0c7-dab2-40de-a259-c32035d4daac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:51.231255 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470de0c7-dab2-40de-a259-c32035d4daac None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:51.231502 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 458/914] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 24 17:08:51 2022] GET /identity/v3/role_assignments?user.id=0ccc6723f7514e52863d102de3839951&scope.project.id=46ab158bc3644c888557781b0630b0b8&role.id=967c5aaa9ba746a7acaa59f3e9691b31 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:51.231606 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470de0c7-dab2-40de-a259-c32035d4daac None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:51.231779 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470de0c7-dab2-40de-a259-c32035d4daac None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:51.232367 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470de0c7-dab2-40de-a259-c32035d4daac None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:51.234929 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470de0c7-dab2-40de-a259-c32035d4daac None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:51.245861 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-470de0c7-dab2-40de-a259-c32035d4daac None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 17:08:51.245861 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:51.245861 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 17:08:51.245861 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 17:08:51.245861 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 17:08:51.245861 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 17:08:51.245861 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 17:08:51.245861 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:08:51.245861 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 17:08:51.245861 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:08:51.245861 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 17:08:51.245861 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:08:51.245861 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:51.245861 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:51.245861 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:51.245861 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:51.245861 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:51.246930 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:51.246930 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:51.246930 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:51.246930 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:51.246930 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:51.246930 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:51.246930 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 17:08:51.246930 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 17:08:51.246930 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 17:08:51.246930 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 17:08:51.246930 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:51.246930 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:51.246930 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:51.246930 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:51.246930 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:51.246930 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:51.247788 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:51.247788 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:51.247788 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:51.247788 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:51.247788 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:51.247788 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:51.247788 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:51.247788 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:51.247788 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:51.247788 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:51.247788 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 17:08:51.247788 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:08:51.247788 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 17:08:51.247788 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:08:51.247788 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 457/915] 10.222.0.18 () {56 vars in 1126 bytes} [Wed Aug 24 17:08:51 2022] GET /identity/v3/services/object-store => generated 93 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:08:51.254058 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3e475f-96ab-4c94-9c26-4a8907b7fe0c None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:51.375157 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3e475f-96ab-4c94-9c26-4a8907b7fe0c None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:51.413304 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1776d0-d3fa-4dcb-8c9a-cc143b7c6244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:51.413984 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1776d0-d3fa-4dcb-8c9a-cc143b7c6244 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:51.414263 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1776d0-d3fa-4dcb-8c9a-cc143b7c6244 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:51.414531 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1776d0-d3fa-4dcb-8c9a-cc143b7c6244 None admin] PATH_INFO: `/v3/services` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:51.415246 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1776d0-d3fa-4dcb-8c9a-cc143b7c6244 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:51.415599 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1776d0-d3fa-4dcb-8c9a-cc143b7c6244 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:51.416970 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1776d0-d3fa-4dcb-8c9a-cc143b7c6244 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:51.422572 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 459/916] 10.222.0.18 () {56 vars in 1122 bytes} [Wed Aug 24 17:08:51 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:51.428264 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470de0c7-dab2-40de-a259-c32035d4daac None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:51.518206 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:51.537417 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470de0c7-dab2-40de-a259-c32035d4daac None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:51.575040 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bc3a5b-8abb-4a89-b005-30c8eb0d0399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:51.575656 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bc3a5b-8abb-4a89-b005-30c8eb0d0399 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:51.575965 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bc3a5b-8abb-4a89-b005-30c8eb0d0399 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:51.576206 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bc3a5b-8abb-4a89-b005-30c8eb0d0399 None admin] PATH_INFO: `/v3/services` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:51.576836 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bc3a5b-8abb-4a89-b005-30c8eb0d0399 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:51.577138 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bc3a5b-8abb-4a89-b005-30c8eb0d0399 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:51.578550 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bc3a5b-8abb-4a89-b005-30c8eb0d0399 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:51.583940 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 458/917] 10.222.0.18 () {56 vars in 1122 bytes} [Wed Aug 24 17:08:51 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:51.589785 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1776d0-d3fa-4dcb-8c9a-cc143b7c6244 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:51.619785 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:51.694072 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1776d0-d3fa-4dcb-8c9a-cc143b7c6244 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:51.738915 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcea6cf4-2a91-4ba5-ad45-96032f9ffd1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:51.739560 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcea6cf4-2a91-4ba5-ad45-96032f9ffd1a None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:51.739843 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcea6cf4-2a91-4ba5-ad45-96032f9ffd1a None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:51.740116 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcea6cf4-2a91-4ba5-ad45-96032f9ffd1a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:51.740829 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcea6cf4-2a91-4ba5-ad45-96032f9ffd1a None admin] RBAC: Adding query filter params (interface=public, service_id=32523190b86f47689ebd4d89862b7a7a, region_id=RegionOne) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:51.741141 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcea6cf4-2a91-4ba5-ad45-96032f9ffd1a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:51.742511 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcea6cf4-2a91-4ba5-ad45-96032f9ffd1a None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:51.749047 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 460/918] 10.222.0.18 () {56 vars in 1251 bytes} [Wed Aug 24 17:08:51 2022] GET /identity/v3/endpoints?service_id=32523190b86f47689ebd4d89862b7a7a&interface=public®ion_id=RegionOne => generated 194 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:51.754874 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bc3a5b-8abb-4a89-b005-30c8eb0d0399 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:51.799654 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a61dfce-e9d5-4e15-9a5b-554d64da3d80 None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:51.799935 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a61dfce-e9d5-4e15-9a5b-554d64da3d80 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:51.800133 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a61dfce-e9d5-4e15-9a5b-554d64da3d80 None None] PATH_INFO: `/` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:51.802210 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 461/919] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:08:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 17:08:51.808085 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eb5406-d78d-4608-881d-99d0bc574370 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:51.809157 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eb5406-d78d-4608-881d-99d0bc574370 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:51.809674 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eb5406-d78d-4608-881d-99d0bc574370 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:51.829738 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-a8eb5406-d78d-4608-881d-99d0bc574370 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:08:51.831442 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-a8eb5406-d78d-4608-881d-99d0bc574370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:08:51.873580 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bc3a5b-8abb-4a89-b005-30c8eb0d0399 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:51.916811 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 462/920] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:08:51 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 17:08:51.920887 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3a0f29-5426-40b0-83c6-f877bf29cc41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:51.921736 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3a0f29-5426-40b0-83c6-f877bf29cc41 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:51.922237 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3a0f29-5426-40b0-83c6-f877bf29cc41 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:51.922724 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3a0f29-5426-40b0-83c6-f877bf29cc41 None admin] PATH_INFO: `/v3/services` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:51.923563 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3a0f29-5426-40b0-83c6-f877bf29cc41 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:51.925124 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3a0f29-5426-40b0-83c6-f877bf29cc41 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:51.932075 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 459/921] 10.222.0.18 () {56 vars in 1087 bytes} [Wed Aug 24 17:08:51 2022] GET /identity/v3/services => generated 1497 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:08:52.192823 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8eb5406-d78d-4608-881d-99d0bc574370 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:52.218826 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:52.298924 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8eb5406-d78d-4608-881d-99d0bc574370 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:52.334723 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535255eb-5b8b-452d-8598-91f8e51bc07e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:52.335392 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535255eb-5b8b-452d-8598-91f8e51bc07e None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:52.335668 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535255eb-5b8b-452d-8598-91f8e51bc07e None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:52.335937 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535255eb-5b8b-452d-8598-91f8e51bc07e None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:52.336518 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535255eb-5b8b-452d-8598-91f8e51bc07e None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:52.337881 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535255eb-5b8b-452d-8598-91f8e51bc07e None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:52.347576 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-535255eb-5b8b-452d-8598-91f8e51bc07e None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 17:08:52.347576 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:52.347576 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 17:08:52.347576 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 17:08:52.347576 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 17:08:52.347576 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 17:08:52.347576 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 17:08:52.347576 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:08:52.347576 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 17:08:52.347576 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:08:52.347576 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 17:08:52.347576 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:08:52.347576 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:52.347576 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:52.347576 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:52.347576 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:52.347576 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:52.348584 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:52.348584 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:52.348584 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:52.348584 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:52.348584 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:52.348584 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:52.348584 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 17:08:52.348584 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 17:08:52.348584 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 17:08:52.348584 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 17:08:52.348584 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:52.348584 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:52.348584 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:52.348584 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:52.348584 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:52.348584 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:52.349412 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:52.349412 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:52.349412 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:52.349412 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:52.349412 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:52.349412 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:52.349412 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:52.349412 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:52.349412 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:52.349412 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:52.349412 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 17:08:52.349412 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:08:52.349412 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 17:08:52.349412 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:08:52.353247 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 463/922] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Aug 24 17:08:52 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:52.354977 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3a0f29-5426-40b0-83c6-f877bf29cc41 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:52.363106 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33938f6f-ac8d-41e0-b48c-c818b5ac3d2c None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:52.363392 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33938f6f-ac8d-41e0-b48c-c818b5ac3d2c None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:52.363678 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33938f6f-ac8d-41e0-b48c-c818b5ac3d2c None None] PATH_INFO: `/` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:52.365243 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 464/923] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:08:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 17:08:52.369444 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:52.370122 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00d3d1a-aa70-473a-850f-c4b2bf87001c None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:52.370404 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00d3d1a-aa70-473a-850f-c4b2bf87001c None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:52.370706 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00d3d1a-aa70-473a-850f-c4b2bf87001c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:52.389533 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-b00d3d1a-aa70-473a-850f-c4b2bf87001c None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:08:52.390780 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-b00d3d1a-aa70-473a-850f-c4b2bf87001c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:08:52.471322 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 465/924] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:08:52 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 17:08:52.482716 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3a0f29-5426-40b0-83c6-f877bf29cc41 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:52.517083 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63381ad-c6d1-4194-95a4-8653b97b9899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:52.517698 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63381ad-c6d1-4194-95a4-8653b97b9899 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:52.517979 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63381ad-c6d1-4194-95a4-8653b97b9899 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:52.518522 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63381ad-c6d1-4194-95a4-8653b97b9899 None admin] PATH_INFO: `/v3/services` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:52.519318 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63381ad-c6d1-4194-95a4-8653b97b9899 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:52.519754 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63381ad-c6d1-4194-95a4-8653b97b9899 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:52.521202 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63381ad-c6d1-4194-95a4-8653b97b9899 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:52.531007 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 460/925] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Aug 24 17:08:52 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:52.532799 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00d3d1a-aa70-473a-850f-c4b2bf87001c None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:52.554921 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:52.637997 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00d3d1a-aa70-473a-850f-c4b2bf87001c None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:52.672928 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eb3b2b-fc5f-43a6-91c1-f63afe21378b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:52.673535 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eb3b2b-fc5f-43a6-91c1-f63afe21378b None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:52.673812 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eb3b2b-fc5f-43a6-91c1-f63afe21378b None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:52.674087 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eb3b2b-fc5f-43a6-91c1-f63afe21378b None admin] PATH_INFO: `/v3/services` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:52.674712 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49eb3b2b-fc5f-43a6-91c1-f63afe21378b None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:52.675035 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49eb3b2b-fc5f-43a6-91c1-f63afe21378b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:52.676529 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49eb3b2b-fc5f-43a6-91c1-f63afe21378b None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:52.686071 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 466/926] 10.222.0.18 () {56 vars in 1114 bytes} [Wed Aug 24 17:08:52 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:52.688004 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63381ad-c6d1-4194-95a4-8653b97b9899 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:52.749761 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eb3b2b-fc5f-43a6-91c1-f63afe21378b None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:52.770212 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:52.805403 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63381ad-c6d1-4194-95a4-8653b97b9899 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:52.842616 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:52.845747 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76dafdc-9636-4811-8896-c4051e8cfd0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:52.846368 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76dafdc-9636-4811-8896-c4051e8cfd0a None admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:52.846645 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76dafdc-9636-4811-8896-c4051e8cfd0a None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:52.846942 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76dafdc-9636-4811-8896-c4051e8cfd0a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:52.847628 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76dafdc-9636-4811-8896-c4051e8cfd0a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:52.848998 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76dafdc-9636-4811-8896-c4051e8cfd0a None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:52.864919 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eb3b2b-fc5f-43a6-91c1-f63afe21378b None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:52.899443 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b577326d-088e-464f-9750-903f18aa23ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:52.900076 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b577326d-088e-464f-9750-903f18aa23ea None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:52.900359 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b577326d-088e-464f-9750-903f18aa23ea None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:52.900637 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b577326d-088e-464f-9750-903f18aa23ea None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:52.901201 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b577326d-088e-464f-9750-903f18aa23ea None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:52.902584 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b577326d-088e-464f-9750-903f18aa23ea None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:52.911453 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-b577326d-088e-464f-9750-903f18aa23ea None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 17:08:52.911453 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:52.911453 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 17:08:52.911453 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 17:08:52.911453 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 17:08:52.911453 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 17:08:52.911453 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 17:08:52.911453 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:08:52.911453 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 17:08:52.911453 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:08:52.911453 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 17:08:52.911453 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:08:52.911453 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:52.911453 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:52.911453 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:52.911453 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:52.912430 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:52.912430 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:52.912430 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:52.912430 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:52.912430 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:52.912430 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:52.912430 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:52.912430 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 17:08:52.912430 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 17:08:52.912430 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 17:08:52.912430 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 17:08:52.912430 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:52.912430 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:52.912430 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:52.912430 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:52.912430 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:52.912430 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:52.913878 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:52.913878 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:52.913878 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:52.913878 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:52.913878 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:52.913878 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:52.913878 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:52.913878 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:52.913878 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:52.913878 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:52.913878 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 17:08:52.913878 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:08:52.913878 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 17:08:52.913878 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:08:52.918105 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 467/927] 10.222.0.18 () {56 vars in 1132 bytes} [Wed Aug 24 17:08:52 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:08:52.918751 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b577326d-088e-464f-9750-903f18aa23ea None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:52.963178 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 461/928] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 24 17:08:52 2022] POST /identity/v3/endpoints => generated 346 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:08:53.041718 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b577326d-088e-464f-9750-903f18aa23ea None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:53.097965 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a881b35f-9922-4bdb-a9d6-c9ec87de6b60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:53.098985 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a881b35f-9922-4bdb-a9d6-c9ec87de6b60 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:53.099495 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a881b35f-9922-4bdb-a9d6-c9ec87de6b60 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:53.099972 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a881b35f-9922-4bdb-a9d6-c9ec87de6b60 None admin] PATH_INFO: `/v3/services` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:53.100919 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a881b35f-9922-4bdb-a9d6-c9ec87de6b60 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:53.101450 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a881b35f-9922-4bdb-a9d6-c9ec87de6b60 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:53.103249 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a881b35f-9922-4bdb-a9d6-c9ec87de6b60 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:53.110130 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 468/929] 10.222.0.18 () {56 vars in 1126 bytes} [Wed Aug 24 17:08:52 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:08:53.115555 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76dafdc-9636-4811-8896-c4051e8cfd0a None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:53.226659 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054c0931-a21a-4081-8a54-962b8f979b8e None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:53.227319 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76dafdc-9636-4811-8896-c4051e8cfd0a None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:53.227554 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054c0931-a21a-4081-8a54-962b8f979b8e None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:53.228050 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054c0931-a21a-4081-8a54-962b8f979b8e None None] PATH_INFO: `/` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:53.229589 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 469/930] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:08:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:08:53.235794 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c5a173-8452-4996-aa0c-dca265fda8b4 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:53.236350 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c5a173-8452-4996-aa0c-dca265fda8b4 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:53.236826 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c5a173-8452-4996-aa0c-dca265fda8b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:53.244205 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:53.257473 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-22c5a173-8452-4996-aa0c-dca265fda8b4 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:08:53.259510 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-22c5a173-8452-4996-aa0c-dca265fda8b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:08:53.264424 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:53.270313 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7279d44c-d91b-4423-8e99-5044392818c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:53.271092 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7279d44c-d91b-4423-8e99-5044392818c0 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:53.271447 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7279d44c-d91b-4423-8e99-5044392818c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:53.271945 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7279d44c-d91b-4423-8e99-5044392818c0 None admin] PATH_INFO: `/v3/services` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:53.272652 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7279d44c-d91b-4423-8e99-5044392818c0 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:53.273228 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7279d44c-d91b-4423-8e99-5044392818c0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:53.274534 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7279d44c-d91b-4423-8e99-5044392818c0 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:53.281630 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 462/931] 10.222.0.18 () {56 vars in 1126 bytes} [Wed Aug 24 17:08:53 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:53.288512 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7279d44c-d91b-4423-8e99-5044392818c0 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:53.344393 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 470/932] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:08:53 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 17:08:53.412721 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7279d44c-d91b-4423-8e99-5044392818c0 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:53.449597 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b521acf-19ad-4a7a-909a-1fe2fd7e38d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:53.450475 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b521acf-19ad-4a7a-909a-1fe2fd7e38d1 None admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:53.450934 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b521acf-19ad-4a7a-909a-1fe2fd7e38d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:53.451355 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b521acf-19ad-4a7a-909a-1fe2fd7e38d1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:53.452179 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b521acf-19ad-4a7a-909a-1fe2fd7e38d1 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:53.453764 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b521acf-19ad-4a7a-909a-1fe2fd7e38d1 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:53.568163 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 463/933] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 24 17:08:53 2022] POST /identity/v3/endpoints => generated 347 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:08:53.633338 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c5a173-8452-4996-aa0c-dca265fda8b4 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:53.741390 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c5a173-8452-4996-aa0c-dca265fda8b4 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:53.786323 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:53.794120 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64699387-259e-4139-8386-0153ccd550e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:53.794288 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64699387-259e-4139-8386-0153ccd550e9 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:53.794573 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64699387-259e-4139-8386-0153ccd550e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:53.794841 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64699387-259e-4139-8386-0153ccd550e9 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:53.806965 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64699387-259e-4139-8386-0153ccd550e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:08:53.811828 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-64699387-259e-4139-8386-0153ccd550e9 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 17:08:53.811828 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:53.811828 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:53.811828 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:53.811828 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:53.811828 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:53.811828 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:53.811828 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:53.811828 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:53.811828 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:53.811828 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:53.811828 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:53.811828 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:08:53.811828 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:08:53.811828 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:08:53.811828 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:08:53.812923 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:08:53.812923 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:08:53.812923 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:08:53.812923 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:08:53.812923 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:53.812923 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:53.812923 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:53.812923 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:53.812923 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:53.812923 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:53.812923 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:53.812923 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:53.812923 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:53.812923 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:53.812923 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:53.812923 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:53.815280 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:53.815280 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:53.815280 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:53.815280 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:53.815280 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:08:53.815280 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:08:53.815280 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:08:53.815280 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:08:53.815280 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:08:53.815280 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:08:53.815280 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 17:08:53.815280 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:08:53.815280 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 471/934] 10.222.0.18 () {56 vars in 1120 bytes} [Wed Aug 24 17:08:53 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:53.824236 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b521acf-19ad-4a7a-909a-1fe2fd7e38d1 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:53.893641 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:53.946761 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0d3b84-fcf3-4029-9fc8-6d94c683ee06 None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:53.947104 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0d3b84-fcf3-4029-9fc8-6d94c683ee06 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:53.947377 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0d3b84-fcf3-4029-9fc8-6d94c683ee06 None None] PATH_INFO: `/` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:53.948711 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 472/935] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:08:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 17:08:53.954012 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada8d413-6cc7-4a77-94e9-d293b7b5b0cb None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:53.954296 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada8d413-6cc7-4a77-94e9-d293b7b5b0cb None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:53.954572 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada8d413-6cc7-4a77-94e9-d293b7b5b0cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:53.971691 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b521acf-19ad-4a7a-909a-1fe2fd7e38d1 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:53.974175 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-ada8d413-6cc7-4a77-94e9-d293b7b5b0cb None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:08:53.975404 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-ada8d413-6cc7-4a77-94e9-d293b7b5b0cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:08:54.043263 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e1cf40-4276-408c-b33d-bca605b5da94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:54.045745 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e1cf40-4276-408c-b33d-bca605b5da94 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:54.047290 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e1cf40-4276-408c-b33d-bca605b5da94 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:54.047742 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e1cf40-4276-408c-b33d-bca605b5da94 None admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:54.051712 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e1cf40-4276-408c-b33d-bca605b5da94 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:54.052191 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e1cf40-4276-408c-b33d-bca605b5da94 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:54.053723 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e1cf40-4276-408c-b33d-bca605b5da94 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:54.073460 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 464/936] 10.222.0.18 () {56 vars in 1115 bytes} [Wed Aug 24 17:08:53 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:54.078963 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e1cf40-4276-408c-b33d-bca605b5da94 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:54.081471 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 473/937] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:08:53 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 17:08:54.192628 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e1cf40-4276-408c-b33d-bca605b5da94 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:54.208208 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:54.234392 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e75897c-9fd7-47ee-a112-39bec492fff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:54.235077 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e75897c-9fd7-47ee-a112-39bec492fff8 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:54.235360 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e75897c-9fd7-47ee-a112-39bec492fff8 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:54.235637 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e75897c-9fd7-47ee-a112-39bec492fff8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:54.247141 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e75897c-9fd7-47ee-a112-39bec492fff8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:54.248590 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e75897c-9fd7-47ee-a112-39bec492fff8 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:54.263997 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-0e75897c-9fd7-47ee-a112-39bec492fff8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:08:54.263997 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:54.263997 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:54.263997 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:54.263997 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:54.263997 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:54.263997 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:54.263997 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:54.263997 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:54.263997 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:54.263997 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:54.263997 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:54.263997 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:08:54.263997 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:08:54.263997 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:08:54.263997 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:08:54.264976 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:54.264976 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:54.264976 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:54.264976 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:54.264976 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:54.264976 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:54.264976 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:54.264976 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:54.264976 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:54.264976 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:54.264976 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:54.264976 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:54.264976 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:54.264976 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:54.264976 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:54.264976 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:54.264976 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:08:54.266295 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:08:54.266295 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:08:54.266295 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:08:54.266295 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 465/938] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 24 17:08:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:54.271352 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada8d413-6cc7-4a77-94e9-d293b7b5b0cb None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:54.373449 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada8d413-6cc7-4a77-94e9-d293b7b5b0cb None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:54.394114 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:54.408768 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffed6ab1-f961-45b7-bb8c-3e6e2d31b9f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:54.409372 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffed6ab1-f961-45b7-bb8c-3e6e2d31b9f9 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:54.409651 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffed6ab1-f961-45b7-bb8c-3e6e2d31b9f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:54.409975 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffed6ab1-f961-45b7-bb8c-3e6e2d31b9f9 None admin] PATH_INFO: `/v3/domains` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:54.410621 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffed6ab1-f961-45b7-bb8c-3e6e2d31b9f9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:54.411007 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffed6ab1-f961-45b7-bb8c-3e6e2d31b9f9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:54.412456 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffed6ab1-f961-45b7-bb8c-3e6e2d31b9f9 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:54.424586 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 474/939] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 24 17:08:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:54.431027 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e75897c-9fd7-47ee-a112-39bec492fff8 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:54.431898 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffed6ab1-f961-45b7-bb8c-3e6e2d31b9f9 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:54.533594 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffed6ab1-f961-45b7-bb8c-3e6e2d31b9f9 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:54.565766 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e75897c-9fd7-47ee-a112-39bec492fff8 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:54.573415 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb6b408-712c-4cec-ad21-04bd453a5722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:54.574743 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb6b408-712c-4cec-ad21-04bd453a5722 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:54.575104 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb6b408-712c-4cec-ad21-04bd453a5722 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:54.575460 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb6b408-712c-4cec-ad21-04bd453a5722 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:54.576574 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb6b408-712c-4cec-ad21-04bd453a5722 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:54.578560 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb6b408-712c-4cec-ad21-04bd453a5722 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:54.591117 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-afb6b408-712c-4cec-ad21-04bd453a5722 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 17:08:54.591117 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:54.591117 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 17:08:54.591117 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 17:08:54.591117 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 17:08:54.591117 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 17:08:54.591117 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 17:08:54.591117 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:08:54.591117 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 17:08:54.591117 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:08:54.591117 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 17:08:54.591117 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:08:54.591117 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:54.591117 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:54.591117 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:54.591117 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:54.591117 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:54.592312 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:54.592312 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:54.592312 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:54.592312 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:54.592312 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:54.592312 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:54.592312 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 17:08:54.592312 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 17:08:54.592312 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 17:08:54.592312 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 17:08:54.592312 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:54.592312 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:54.592312 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:54.592312 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:54.592312 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:54.592312 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:54.593157 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:54.593157 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:54.593157 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:54.593157 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:54.593157 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:54.593157 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:54.593157 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:54.593157 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:54.593157 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:54.593157 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:54.593157 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 17:08:54.593157 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:08:54.593157 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 17:08:54.593157 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:08:54.594309 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 475/940] 10.222.0.18 () {56 vars in 1126 bytes} [Wed Aug 24 17:08:54 2022] GET /identity/v3/services/object-store => generated 93 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:54.599727 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb6b408-712c-4cec-ad21-04bd453a5722 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:54.607891 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cb8455-c81c-409f-94fc-349d689390ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:54.608600 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cb8455-c81c-409f-94fc-349d689390ea None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:54.608959 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cb8455-c81c-409f-94fc-349d689390ea None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:54.609296 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cb8455-c81c-409f-94fc-349d689390ea None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:54.617375 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cb8455-c81c-409f-94fc-349d689390ea None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:54.618813 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cb8455-c81c-409f-94fc-349d689390ea None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:54.630739 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-75cb8455-c81c-409f-94fc-349d689390ea None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 17:08:54.630739 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:54.630739 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:54.630739 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:54.630739 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:54.630739 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:54.630739 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:54.630739 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:54.630739 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:54.630739 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:54.630739 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:54.630739 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:54.630739 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:08:54.630739 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:08:54.630739 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:08:54.630739 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:08:54.632073 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:54.632073 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:54.632073 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:08:54.632073 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:08:54.632073 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:08:54.632073 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:08:54.632073 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:54.632073 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:54.632073 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:54.632073 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:54.632073 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:54.632073 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:54.632073 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:54.632073 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:54.632073 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:54.632073 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:54.632073 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:54.632073 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:54.633965 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:54.633965 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:54.633965 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:08:54.633965 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:08:54.633965 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:08:54.633965 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:08:54.633965 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:08:54.633965 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:08:54.633965 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 17:08:54.633965 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:08:54.638329 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 466/941] 10.222.0.18 () {56 vars in 1117 bytes} [Wed Aug 24 17:08:54 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:54.639260 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cb8455-c81c-409f-94fc-349d689390ea None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:54.660416 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:54.666241 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:54.736613 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb6b408-712c-4cec-ad21-04bd453a5722 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:54.747138 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cb8455-c81c-409f-94fc-349d689390ea None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:54.785954 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0120617e-add0-401f-ab9a-2b518c84cd08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:54.786557 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0120617e-add0-401f-ab9a-2b518c84cd08 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:54.786912 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0120617e-add0-401f-ab9a-2b518c84cd08 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:54.787260 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0120617e-add0-401f-ab9a-2b518c84cd08 None admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:54.788052 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0120617e-add0-401f-ab9a-2b518c84cd08 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:54.788422 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0120617e-add0-401f-ab9a-2b518c84cd08 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:54.788841 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca829264-d8cf-4a79-8cd7-6e9c0a1bcac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:54.789860 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca829264-d8cf-4a79-8cd7-6e9c0a1bcac7 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:54.790042 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0120617e-add0-401f-ab9a-2b518c84cd08 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:54.790334 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca829264-d8cf-4a79-8cd7-6e9c0a1bcac7 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:54.790828 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca829264-d8cf-4a79-8cd7-6e9c0a1bcac7 None admin] PATH_INFO: `/v3/services` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:54.792396 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca829264-d8cf-4a79-8cd7-6e9c0a1bcac7 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:54.792710 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca829264-d8cf-4a79-8cd7-6e9c0a1bcac7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:54.795232 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca829264-d8cf-4a79-8cd7-6e9c0a1bcac7 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:54.803606 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 476/942] 10.222.0.18 () {56 vars in 1122 bytes} [Wed Aug 24 17:08:54 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:08:54.809760 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca829264-d8cf-4a79-8cd7-6e9c0a1bcac7 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:54.820479 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 467/943] 10.222.0.18 () {56 vars in 1149 bytes} [Wed Aug 24 17:08:54 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:54.821343 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0120617e-add0-401f-ab9a-2b518c84cd08 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:54.911489 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:54.931248 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca829264-d8cf-4a79-8cd7-6e9c0a1bcac7 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:54.931792 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0120617e-add0-401f-ab9a-2b518c84cd08 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:54.941770 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:54.968060 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0723df38-9861-4901-9484-0594535188b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:54.968858 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0723df38-9861-4901-9484-0594535188b4 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:54.969272 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0723df38-9861-4901-9484-0594535188b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:54.969617 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0723df38-9861-4901-9484-0594535188b4 None admin] PATH_INFO: `/v3/services` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:54.970101 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0dd7e5-bd3b-4cd5-8965-47c240aadca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:54.970580 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0723df38-9861-4901-9484-0594535188b4 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:54.971000 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0723df38-9861-4901-9484-0594535188b4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:54.971383 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0dd7e5-bd3b-4cd5-8965-47c240aadca8 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:54.971841 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0dd7e5-bd3b-4cd5-8965-47c240aadca8 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:54.972286 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0dd7e5-bd3b-4cd5-8965-47c240aadca8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:54.972679 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0723df38-9861-4901-9484-0594535188b4 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:54.979705 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 477/944] 10.222.0.18 () {56 vars in 1122 bytes} [Wed Aug 24 17:08:54 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:54.985601 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0723df38-9861-4901-9484-0594535188b4 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:54.986209 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0dd7e5-bd3b-4cd5-8965-47c240aadca8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:54.987951 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0dd7e5-bd3b-4cd5-8965-47c240aadca8 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:55.004139 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-bc0dd7e5-bd3b-4cd5-8965-47c240aadca8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:08:55.004139 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:55.004139 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:55.004139 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:55.004139 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:55.004139 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:55.004139 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:55.004139 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:55.004139 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:55.004139 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:55.004139 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:55.004139 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:55.004139 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:08:55.004139 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:08:55.004139 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:08:55.004139 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:08:55.005166 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:55.005166 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:55.005166 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:55.005166 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:55.005166 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:55.005166 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:55.005166 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:55.005166 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:55.005166 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:55.005166 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:55.005166 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:55.005166 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:55.005166 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:55.005166 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:55.005166 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:55.005166 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:55.005166 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:08:55.006077 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:08:55.006077 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:08:55.006077 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:08:55.008229 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 468/945] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 24 17:08:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:55.015009 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0dd7e5-bd3b-4cd5-8965-47c240aadca8 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:55.092152 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0723df38-9861-4901-9484-0594535188b4 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:55.122772 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0dd7e5-bd3b-4cd5-8965-47c240aadca8 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:55.126860 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ee772b-a37f-4dba-a946-64b9c68851bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:55.127636 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ee772b-a37f-4dba-a946-64b9c68851bb None admin] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:55.128071 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ee772b-a37f-4dba-a946-64b9c68851bb None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:55.128482 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ee772b-a37f-4dba-a946-64b9c68851bb None admin] PATH_INFO: `/v3/endpoints` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:55.129318 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ee772b-a37f-4dba-a946-64b9c68851bb None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:55.130922 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ee772b-a37f-4dba-a946-64b9c68851bb None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:55.157380 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35f3c6b-7b02-4866-ae33-89886ba51b23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:55.157986 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35f3c6b-7b02-4866-ae33-89886ba51b23 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:55.158271 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35f3c6b-7b02-4866-ae33-89886ba51b23 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:55.158548 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35f3c6b-7b02-4866-ae33-89886ba51b23 None admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:55.159214 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35f3c6b-7b02-4866-ae33-89886ba51b23 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:55.159521 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35f3c6b-7b02-4866-ae33-89886ba51b23 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:55.160879 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35f3c6b-7b02-4866-ae33-89886ba51b23 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:55.162291 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:55.172808 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 469/946] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 24 17:08:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:08:55.174066 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055ac407-9f8a-49a0-83f1-4ef30ec4b981 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:55.174357 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055ac407-9f8a-49a0-83f1-4ef30ec4b981 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:55.174630 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055ac407-9f8a-49a0-83f1-4ef30ec4b981 None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:55.176028 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 470/947] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:08:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:08:55.179722 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055ac407-9f8a-49a0-83f1-4ef30ec4b981 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:55.224157 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 478/948] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 24 17:08:54 2022] POST /identity/v3/endpoints => generated 349 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:08:55.225394 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b5ebec-54a7-4ccb-88cc-cc978c061014 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:55.225691 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b5ebec-54a7-4ccb-88cc-cc978c061014 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:55.225946 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b5ebec-54a7-4ccb-88cc-cc978c061014 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:55.247183 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-25b5ebec-54a7-4ccb-88cc-cc978c061014 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:08:55.248364 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-25b5ebec-54a7-4ccb-88cc-cc978c061014 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:08:55.277890 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:55.333519 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055ac407-9f8a-49a0-83f1-4ef30ec4b981 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:55.338282 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 479/949] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:08:55 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:08:55.375532 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e499d75-3192-47fb-8dd8-ea75c6581f8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:55.376753 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e499d75-3192-47fb-8dd8-ea75c6581f8f None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:55.377253 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e499d75-3192-47fb-8dd8-ea75c6581f8f None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:55.377736 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e499d75-3192-47fb-8dd8-ea75c6581f8f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:55.385436 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e499d75-3192-47fb-8dd8-ea75c6581f8f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:55.387786 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e499d75-3192-47fb-8dd8-ea75c6581f8f None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:55.399493 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-1e499d75-3192-47fb-8dd8-ea75c6581f8f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:08:55.399493 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:55.399493 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:55.399493 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:55.399493 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:55.399493 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:55.399493 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:55.399493 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:55.399493 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:55.399493 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:55.399493 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:55.399493 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:55.399493 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:08:55.399493 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:08:55.399493 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:08:55.400417 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:08:55.400417 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:55.400417 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:55.400417 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:55.400417 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:55.400417 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:55.400417 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:55.400417 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:55.400417 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:55.400417 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:55.400417 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:55.400417 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:55.400417 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:55.400417 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:55.400417 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:55.400417 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:55.400417 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:55.401346 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:08:55.401346 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:08:55.401346 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:08:55.401346 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:08:55.401346 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:08:55.401346 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:08:55.401346 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:08:55.401346 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:08:55.401767 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 471/950] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 24 17:08:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:55.407072 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b5ebec-54a7-4ccb-88cc-cc978c061014 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:55.509344 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b5ebec-54a7-4ccb-88cc-cc978c061014 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:55.555672 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7451d567-e633-460e-9b47-edd4ed762940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:55.556645 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7451d567-e633-460e-9b47-edd4ed762940 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:55.557151 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7451d567-e633-460e-9b47-edd4ed762940 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:55.557631 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7451d567-e633-460e-9b47-edd4ed762940 None admin] PATH_INFO: `/v3/projects` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:55.559099 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7451d567-e633-460e-9b47-edd4ed762940 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:55.559605 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7451d567-e633-460e-9b47-edd4ed762940 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:55.561260 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7451d567-e633-460e-9b47-edd4ed762940 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:55.574639 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 480/951] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Aug 24 17:08:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:55.587003 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e499d75-3192-47fb-8dd8-ea75c6581f8f None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:55.617877 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7451d567-e633-460e-9b47-edd4ed762940 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:55.643374 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:55.683945 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:55.711675 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e499d75-3192-47fb-8dd8-ea75c6581f8f None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:55.728725 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7451d567-e633-460e-9b47-edd4ed762940 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:55.756635 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df21808e-98c2-4d21-a29f-5d71f688b096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:55.757406 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df21808e-98c2-4d21-a29f-5d71f688b096 None admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:55.757746 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df21808e-98c2-4d21-a29f-5d71f688b096 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:55.758080 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df21808e-98c2-4d21-a29f-5d71f688b096 None admin] PATH_INFO: `/v3/projects/46ab158bc3644c888557781b0630b0b8/users/0ccc6723f7514e52863d102de3839951/roles/967c5aaa9ba746a7acaa59f3e9691b31` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:55.760591 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df21808e-98c2-4d21-a29f-5d71f688b096 None admin] RBAC: Authorizing `identity:create_grant(project_id=46ab158bc3644c888557781b0630b0b8, user_id=0ccc6723f7514e52863d102de3839951, role_id=967c5aaa9ba746a7acaa59f3e9691b31)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:55.762207 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df21808e-98c2-4d21-a29f-5d71f688b096 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:55.771448 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294fe6ab-8f94-43b8-8cd0-e687c0413cfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:55.772057 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294fe6ab-8f94-43b8-8cd0-e687c0413cfb None admin] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:55.772336 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294fe6ab-8f94-43b8-8cd0-e687c0413cfb None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:55.772606 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294fe6ab-8f94-43b8-8cd0-e687c0413cfb None admin] PATH_INFO: `/v3/projects` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:55.773241 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294fe6ab-8f94-43b8-8cd0-e687c0413cfb None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:55.774654 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294fe6ab-8f94-43b8-8cd0-e687c0413cfb None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:55.777310 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: WARNING keystone.server.flask.common [None req-294fe6ab-8f94-43b8-8cd0-e687c0413cfb None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 24 17:08:55.828874 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.sql.core [None req-294fe6ab-8f94-43b8-8cd0-e687c0413cfb None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 24 17:08:55.828874 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 24 17:08:55.828874 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [parameters: {'id': '2b2fecb7dbdc4e05aa90e54f466142b6', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 24 17:08:55.828874 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81731) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 24 17:08:55.835307 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 472/952] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Aug 24 17:08:55 2022] PUT /identity/v3/projects/46ab158bc3644c888557781b0630b0b8/users/0ccc6723f7514e52863d102de3839951/roles/967c5aaa9ba746a7acaa59f3e9691b31 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:08:55.836670 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50acaf37-3c11-45a3-8630-13ea17f78f32 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:55.836999 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50acaf37-3c11-45a3-8630-13ea17f78f32 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:55.837270 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50acaf37-3c11-45a3-8630-13ea17f78f32 None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:55.838834 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 473/953] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:08:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:08:55.847927 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-294fe6ab-8f94-43b8-8cd0-e687c0413cfb None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 2b2fecb7dbdc4e05aa90e54f466142b6.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 2b2fecb7dbdc4e05aa90e54f466142b6. Aug 24 17:08:55.847927 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:55.847927 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 17:08:55.847927 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 17:08:55.847927 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 17:08:55.847927 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 17:08:55.847927 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 17:08:55.847927 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 17:08:55.847927 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 17:08:55.847927 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application conn.query(q) Aug 24 17:08:55.847927 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 17:08:55.847927 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 17:08:55.847927 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 17:08:55.848972 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application result.read() Aug 24 17:08:55.848972 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 17:08:55.848972 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 17:08:55.848972 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 17:08:55.848972 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 17:08:55.848972 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 17:08:55.848972 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 17:08:55.848972 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 17:08:55.848972 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 17:08:55.848972 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 24 17:08:55.848972 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:08:55.848972 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 24 17:08:55.848972 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:08:55.848972 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:55.848972 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 24 17:08:55.848972 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 24 17:08:55.848972 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 24 17:08:55.848972 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 24 17:08:55.850113 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 17:08:55.850113 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application next(self.gen) Aug 24 17:08:55.850113 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 24 17:08:55.850113 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application yield resource Aug 24 17:08:55.850113 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 17:08:55.850113 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application next(self.gen) Aug 24 17:08:55.850113 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 24 17:08:55.850113 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application self.session.rollback() Aug 24 17:08:55.850113 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 24 17:08:55.850113 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application self.force_reraise() Aug 24 17:08:55.850113 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 24 17:08:55.850113 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise self.value Aug 24 17:08:55.850113 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 24 17:08:55.850113 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 24 17:08:55.850113 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 24 17:08:55.850113 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application session.commit() Aug 24 17:08:55.850113 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 24 17:08:55.851213 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 24 17:08:55.851213 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 24 17:08:55.851213 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application self._prepare_impl() Aug 24 17:08:55.851213 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 24 17:08:55.851213 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application self.session.flush() Aug 24 17:08:55.851213 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 24 17:08:55.851213 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application self._flush(objects) Aug 24 17:08:55.851213 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 24 17:08:55.851213 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 24 17:08:55.851213 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 24 17:08:55.851213 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application compat.raise_( Aug 24 17:08:55.851213 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 17:08:55.851213 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception Aug 24 17:08:55.851213 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 24 17:08:55.851213 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application flush_context.execute() Aug 24 17:08:55.851213 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 24 17:08:55.851213 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rec.execute(self) Aug 24 17:08:55.852322 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 24 17:08:55.852322 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 24 17:08:55.852322 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 24 17:08:55.852322 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 24 17:08:55.852322 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 24 17:08:55.852322 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 24 17:08:55.852322 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 24 17:08:55.852322 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 24 17:08:55.852322 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 24 17:08:55.852322 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 24 17:08:55.852322 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 24 17:08:55.852322 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 24 17:08:55.852322 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 24 17:08:55.852322 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 24 17:08:55.852322 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 24 17:08:55.853290 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 24 17:08:55.853290 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 17:08:55.853290 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception Aug 24 17:08:55.853290 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 17:08:55.853290 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 17:08:55.853290 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 17:08:55.853290 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 17:08:55.853290 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 17:08:55.853290 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 17:08:55.853290 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 17:08:55.853290 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application conn.query(q) Aug 24 17:08:55.853290 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 17:08:55.853290 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 17:08:55.853290 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 17:08:55.853290 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application result.read() Aug 24 17:08:55.853290 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 17:08:55.853290 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 17:08:55.854376 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 17:08:55.854376 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 17:08:55.854376 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 17:08:55.854376 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 17:08:55.854376 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 17:08:55.854376 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 17:08:55.854376 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 24 17:08:55.854376 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 24 17:08:55.854376 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [parameters: {'id': '2b2fecb7dbdc4e05aa90e54f466142b6', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 24 17:08:55.854376 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 24 17:08:55.854376 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:08:55.854376 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 17:08:55.854376 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:08:55.854376 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:55.854376 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 24 17:08:55.855933 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 24 17:08:55.855933 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 24 17:08:55.855933 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 24 17:08:55.855933 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 24 17:08:55.855933 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:08:55.855933 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 17:08:55.855933 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:08:55.855933 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:55.855933 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:55.855933 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:55.855933 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:55.855933 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:55.855933 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:55.855933 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:55.855933 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:55.855933 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:55.855933 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:55.857256 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:55.857256 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 24 17:08:55.857256 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 24 17:08:55.857256 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:55.857256 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:55.857256 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 24 17:08:55.857256 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 24 17:08:55.857256 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 2b2fecb7dbdc4e05aa90e54f466142b6. Aug 24 17:08:55.857256 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:08:55.857903 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6300f00a-e09e-4d5a-9b42-3926a9cc531c None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:55.857903 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6300f00a-e09e-4d5a-9b42-3926a9cc531c None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:55.859645 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6300f00a-e09e-4d5a-9b42-3926a9cc531c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:55.860071 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 481/954] 10.222.0.18 () {60 vars in 1140 bytes} [Wed Aug 24 17:08:55 2022] POST /identity/v3/projects => generated 262 bytes in 243 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:08:55.868461 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294fe6ab-8f94-43b8-8cd0-e687c0413cfb None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:55.883226 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-6300f00a-e09e-4d5a-9b42-3926a9cc531c None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:08:55.884634 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-6300f00a-e09e-4d5a-9b42-3926a9cc531c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:08:55.963102 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:55.977495 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 474/955] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:08:55 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 17:08:56.003054 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294fe6ab-8f94-43b8-8cd0-e687c0413cfb None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:56.040389 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3488e0a0-c316-47db-b11a-162b8e1cb355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:56.041359 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3488e0a0-c316-47db-b11a-162b8e1cb355 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:56.041874 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3488e0a0-c316-47db-b11a-162b8e1cb355 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:56.042342 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3488e0a0-c316-47db-b11a-162b8e1cb355 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:56.051641 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3488e0a0-c316-47db-b11a-162b8e1cb355 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:56.053607 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3488e0a0-c316-47db-b11a-162b8e1cb355 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:56.065056 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-3488e0a0-c316-47db-b11a-162b8e1cb355 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:08:56.065056 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:56.065056 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:56.065056 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:56.065056 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:56.065056 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:56.065056 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:56.065056 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:56.065056 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:56.065056 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:56.065056 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:56.065056 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:56.065056 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:08:56.065056 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:08:56.065056 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:08:56.065932 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:08:56.065932 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:56.065932 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:56.065932 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:56.065932 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:56.065932 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:56.065932 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:56.065932 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:56.065932 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:56.065932 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:56.065932 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:56.065932 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:56.065932 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:56.065932 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:56.065932 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:56.065932 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:56.065932 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:56.066958 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:08:56.066958 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:08:56.066958 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:08:56.066958 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:08:56.066958 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:08:56.066958 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:08:56.066958 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:08:56.066958 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:08:56.067829 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 482/956] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 24 17:08:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:08:56.073369 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6300f00a-e09e-4d5a-9b42-3926a9cc531c None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:56.126884 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:56.179594 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6300f00a-e09e-4d5a-9b42-3926a9cc531c None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:56.213373 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdada04-bdb0-4d09-9018-9e938445c8fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:56.214209 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdada04-bdb0-4d09-9018-9e938445c8fa None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:56.215198 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdada04-bdb0-4d09-9018-9e938445c8fa None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:56.215807 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdada04-bdb0-4d09-9018-9e938445c8fa None admin] PATH_INFO: `/v3/projects` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:56.216623 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdada04-bdb0-4d09-9018-9e938445c8fa None admin] RBAC: Adding query filter params (name=service) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:56.217169 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdada04-bdb0-4d09-9018-9e938445c8fa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:56.219254 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdada04-bdb0-4d09-9018-9e938445c8fa None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:56.237771 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 475/957] 10.222.0.18 () {56 vars in 1112 bytes} [Wed Aug 24 17:08:56 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:56.248569 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3488e0a0-c316-47db-b11a-162b8e1cb355 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:56.368288 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3488e0a0-c316-47db-b11a-162b8e1cb355 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:56.403673 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:56.409773 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bbfa03-da54-4f7e-a416-49d1c25d344c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:56.410421 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bbfa03-da54-4f7e-a416-49d1c25d344c None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:56.410701 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bbfa03-da54-4f7e-a416-49d1c25d344c None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:56.411102 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bbfa03-da54-4f7e-a416-49d1c25d344c None admin] PATH_INFO: `/v3/services/compute` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:56.411693 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bbfa03-da54-4f7e-a416-49d1c25d344c None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:56.413057 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bbfa03-da54-4f7e-a416-49d1c25d344c None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:56.422140 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-a8bbfa03-da54-4f7e-a416-49d1c25d344c None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 17:08:56.422140 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:56.422140 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 17:08:56.422140 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 17:08:56.422140 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 17:08:56.422140 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 17:08:56.422140 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 17:08:56.422140 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:08:56.422140 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 17:08:56.422140 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:08:56.422140 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 17:08:56.422140 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:08:56.422140 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:56.422140 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:56.422140 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:56.422140 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:56.422140 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:56.423212 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:56.423212 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:56.423212 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:56.423212 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:56.423212 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:56.423212 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:56.423212 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 17:08:56.423212 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 17:08:56.423212 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 17:08:56.423212 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 17:08:56.423212 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:56.423212 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:56.423212 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:56.423212 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:56.423212 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:56.423212 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:56.424145 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:56.424145 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:56.424145 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:56.424145 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:56.424145 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:56.424145 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:56.424145 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:56.424145 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:56.424145 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:56.424145 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:56.424145 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 17:08:56.424145 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:08:56.424145 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 17:08:56.424145 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:08:56.424145 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 483/958] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 24 17:08:56 2022] GET /identity/v3/services/compute => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:56.429323 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdada04-bdb0-4d09-9018-9e938445c8fa None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:56.539725 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:56.543450 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdada04-bdb0-4d09-9018-9e938445c8fa None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:56.580776 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44492c28-c656-4d0a-9753-462402c8362c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:56.581395 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44492c28-c656-4d0a-9753-462402c8362c None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:56.581679 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44492c28-c656-4d0a-9753-462402c8362c None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:56.581951 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44492c28-c656-4d0a-9753-462402c8362c None admin] PATH_INFO: `/v3/services` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:56.582581 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44492c28-c656-4d0a-9753-462402c8362c None admin] RBAC: Adding query filter params (name=compute) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:56.582886 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44492c28-c656-4d0a-9753-462402c8362c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:56.584348 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44492c28-c656-4d0a-9753-462402c8362c None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:56.590003 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 476/959] 10.222.0.18 () {56 vars in 1112 bytes} [Wed Aug 24 17:08:56 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:56.595630 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bbfa03-da54-4f7e-a416-49d1c25d344c None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:56.699797 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bbfa03-da54-4f7e-a416-49d1c25d344c None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:56.734629 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c7cfc1-34e3-42c9-bbf7-e8d27e02edc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:56.735274 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c7cfc1-34e3-42c9-bbf7-e8d27e02edc5 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:56.735537 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c7cfc1-34e3-42c9-bbf7-e8d27e02edc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:56.735795 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c7cfc1-34e3-42c9-bbf7-e8d27e02edc5 None admin] PATH_INFO: `/v3/services` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:56.736418 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c7cfc1-34e3-42c9-bbf7-e8d27e02edc5 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:56.736721 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c7cfc1-34e3-42c9-bbf7-e8d27e02edc5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:56.738070 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c7cfc1-34e3-42c9-bbf7-e8d27e02edc5 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:56.746380 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 484/960] 10.222.0.18 () {56 vars in 1112 bytes} [Wed Aug 24 17:08:56 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:57.300367 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ecd6b9-137d-4d5d-ba5f-59887e97d185 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:57.300877 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ecd6b9-137d-4d5d-ba5f-59887e97d185 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:57.301270 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ecd6b9-137d-4d5d-ba5f-59887e97d185 None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:57.303071 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 477/961] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:08:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:08:57.309024 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947047b7-6783-4ab9-977e-96ca4c4bc954 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:57.309420 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947047b7-6783-4ab9-977e-96ca4c4bc954 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:57.309796 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947047b7-6783-4ab9-977e-96ca4c4bc954 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:57.328628 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-947047b7-6783-4ab9-977e-96ca4c4bc954 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:08:57.330146 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-947047b7-6783-4ab9-977e-96ca4c4bc954 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:08:57.387681 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:57.402476 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 485/962] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:08:57 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:08:57.689446 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ecd6b9-137d-4d5d-ba5f-59887e97d185 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:57.797858 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ecd6b9-137d-4d5d-ba5f-59887e97d185 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:57.834873 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f53a2b-b4dc-4ae0-b0d6-8306c6d959fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:57.835553 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f53a2b-b4dc-4ae0-b0d6-8306c6d959fe None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:57.835829 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f53a2b-b4dc-4ae0-b0d6-8306c6d959fe None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:57.836099 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f53a2b-b4dc-4ae0-b0d6-8306c6d959fe None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:57.836692 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f53a2b-b4dc-4ae0-b0d6-8306c6d959fe None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:57.838102 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f53a2b-b4dc-4ae0-b0d6-8306c6d959fe None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:57.847905 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-60f53a2b-b4dc-4ae0-b0d6-8306c6d959fe None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 17:08:57.847905 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:57.847905 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 17:08:57.847905 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 17:08:57.847905 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 17:08:57.847905 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 17:08:57.847905 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 17:08:57.847905 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:08:57.847905 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 17:08:57.847905 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:08:57.847905 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 17:08:57.847905 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:08:57.847905 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:57.847905 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:57.847905 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:57.847905 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:57.847905 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:57.848969 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:57.848969 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:57.848969 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:57.848969 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:57.848969 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:57.848969 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:57.848969 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 17:08:57.848969 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 17:08:57.848969 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 17:08:57.848969 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 17:08:57.848969 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:57.848969 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:57.848969 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:57.848969 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:57.848969 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:57.848969 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:57.849825 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:57.849825 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:57.849825 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:57.849825 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:57.849825 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:57.849825 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:57.849825 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:57.849825 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:57.849825 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:57.849825 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:57.849825 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 17:08:57.849825 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:08:57.849825 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 17:08:57.849825 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:08:57.849825 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 478/963] 10.222.0.18 () {56 vars in 1126 bytes} [Wed Aug 24 17:08:57 2022] GET /identity/v3/services/object-store => generated 93 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:57.855322 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947047b7-6783-4ab9-977e-96ca4c4bc954 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:57.917006 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6b5430-b041-401b-b320-5a07729bad6a None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:57.918176 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6b5430-b041-401b-b320-5a07729bad6a None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:57.918660 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6b5430-b041-401b-b320-5a07729bad6a None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:57.920507 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 479/964] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:08:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:08:57.925588 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3062b4-9791-48a1-9052-de75c5f71b95 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:57.926133 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3062b4-9791-48a1-9052-de75c5f71b95 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:57.927294 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3062b4-9791-48a1-9052-de75c5f71b95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:57.946716 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-2c3062b4-9791-48a1-9052-de75c5f71b95 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:08:57.948234 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-2c3062b4-9791-48a1-9052-de75c5f71b95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:08:57.961040 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947047b7-6783-4ab9-977e-96ca4c4bc954 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:57.988867 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:58.003033 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf758175-9077-4f9c-bb11-d5e2a13649d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:58.004028 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf758175-9077-4f9c-bb11-d5e2a13649d9 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:58.004511 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf758175-9077-4f9c-bb11-d5e2a13649d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:58.004993 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf758175-9077-4f9c-bb11-d5e2a13649d9 None admin] PATH_INFO: `/v3/services` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:58.005804 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf758175-9077-4f9c-bb11-d5e2a13649d9 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:58.006274 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf758175-9077-4f9c-bb11-d5e2a13649d9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:58.007840 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf758175-9077-4f9c-bb11-d5e2a13649d9 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:58.013004 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 486/965] 10.222.0.18 () {56 vars in 1122 bytes} [Wed Aug 24 17:08:57 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:58.019560 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf758175-9077-4f9c-bb11-d5e2a13649d9 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:58.047087 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 480/966] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:08:57 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 17:08:58.083387 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:58.126930 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf758175-9077-4f9c-bb11-d5e2a13649d9 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:58.162195 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c74b655-5fe2-4f02-8ece-30d026474b8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:58.163248 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c74b655-5fe2-4f02-8ece-30d026474b8e None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:58.163738 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c74b655-5fe2-4f02-8ece-30d026474b8e None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:58.164236 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c74b655-5fe2-4f02-8ece-30d026474b8e None admin] PATH_INFO: `/v3/services` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:58.165071 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c74b655-5fe2-4f02-8ece-30d026474b8e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:58.165588 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c74b655-5fe2-4f02-8ece-30d026474b8e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:58.167168 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c74b655-5fe2-4f02-8ece-30d026474b8e None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:58.173302 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 487/967] 10.222.0.18 () {56 vars in 1122 bytes} [Wed Aug 24 17:08:58 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:58.179413 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3062b4-9791-48a1-9052-de75c5f71b95 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:58.284988 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3062b4-9791-48a1-9052-de75c5f71b95 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:58.312953 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c74b655-5fe2-4f02-8ece-30d026474b8e None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:58.324481 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97219084-28f0-475b-ad43-de6bf0ba0b9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:58.325415 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97219084-28f0-475b-ad43-de6bf0ba0b9e None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:58.325706 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97219084-28f0-475b-ad43-de6bf0ba0b9e None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:58.325984 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97219084-28f0-475b-ad43-de6bf0ba0b9e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:58.327005 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97219084-28f0-475b-ad43-de6bf0ba0b9e None admin] RBAC: Adding query filter params (interface=admin, service_id=32523190b86f47689ebd4d89862b7a7a, region_id=RegionOne) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:58.327324 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97219084-28f0-475b-ad43-de6bf0ba0b9e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:58.329233 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97219084-28f0-475b-ad43-de6bf0ba0b9e None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:58.335703 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 481/968] 10.222.0.18 () {56 vars in 1249 bytes} [Wed Aug 24 17:08:58 2022] GET /identity/v3/endpoints?service_id=32523190b86f47689ebd4d89862b7a7a&interface=admin®ion_id=RegionOne => generated 193 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:58.337062 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b02c57b-91a2-4b93-a88a-1988fbb386dd None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:58.337349 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b02c57b-91a2-4b93-a88a-1988fbb386dd None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:58.337624 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b02c57b-91a2-4b93-a88a-1988fbb386dd None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:58.340188 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 482/969] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:08:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:08:58.345953 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881c4998-ae49-4a6d-adf0-861947e11a39 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:58.346244 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881c4998-ae49-4a6d-adf0-861947e11a39 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:58.346522 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881c4998-ae49-4a6d-adf0-861947e11a39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:58.367180 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-881c4998-ae49-4a6d-adf0-861947e11a39 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:08:58.368354 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-881c4998-ae49-4a6d-adf0-861947e11a39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:08:58.386942 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:58.421751 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c74b655-5fe2-4f02-8ece-30d026474b8e None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:58.439606 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 483/970] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:08:58 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:08:58.440263 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:58.447804 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881c4998-ae49-4a6d-adf0-861947e11a39 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:58.466995 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a574b5eb-0b09-4897-b213-fe9ed94d15cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:58.468248 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a574b5eb-0b09-4897-b213-fe9ed94d15cf None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:58.468754 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a574b5eb-0b09-4897-b213-fe9ed94d15cf None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:58.469404 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a574b5eb-0b09-4897-b213-fe9ed94d15cf None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:58.484611 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a574b5eb-0b09-4897-b213-fe9ed94d15cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:08:58.491455 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-a574b5eb-0b09-4897-b213-fe9ed94d15cf None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 17:08:58.491455 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:58.491455 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:58.491455 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:58.491455 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:58.491455 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:58.491455 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:58.491455 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:58.491455 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:58.491455 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:58.491455 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:58.491455 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:58.491455 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:08:58.491455 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:08:58.491455 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:08:58.491455 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:08:58.492566 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:08:58.492566 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:08:58.492566 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:08:58.492566 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:08:58.492566 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:58.492566 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:58.492566 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:58.492566 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:58.492566 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:58.492566 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:58.492566 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:58.492566 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:58.492566 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:58.492566 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:58.492566 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:58.492566 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:58.493424 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:58.493424 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:58.493424 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:58.493424 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:58.493424 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:08:58.493424 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:08:58.493424 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:08:58.493424 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:08:58.493424 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:08:58.493424 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:08:58.493424 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 17:08:58.493424 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:08:58.496186 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 488/971] 10.222.0.18 () {56 vars in 1120 bytes} [Wed Aug 24 17:08:58 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:08:58.501934 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a574b5eb-0b09-4897-b213-fe9ed94d15cf None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:58.578368 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881c4998-ae49-4a6d-adf0-861947e11a39 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:58.617475 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8881a3a6-af04-4f93-b302-37b238358c41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:58.618250 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8881a3a6-af04-4f93-b302-37b238358c41 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:58.618632 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8881a3a6-af04-4f93-b302-37b238358c41 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:58.619042 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8881a3a6-af04-4f93-b302-37b238358c41 None admin] PATH_INFO: `/v3/services` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:58.619750 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8881a3a6-af04-4f93-b302-37b238358c41 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:58.621253 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8881a3a6-af04-4f93-b302-37b238358c41 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:58.630974 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a574b5eb-0b09-4897-b213-fe9ed94d15cf None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:58.635607 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 484/972] 10.222.0.18 () {56 vars in 1087 bytes} [Wed Aug 24 17:08:58 2022] GET /identity/v3/services => generated 1497 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:08:58.672345 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257aa9a5-8220-439a-8397-5c2e07aaa9ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:58.673259 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257aa9a5-8220-439a-8397-5c2e07aaa9ca None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:58.673721 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257aa9a5-8220-439a-8397-5c2e07aaa9ca None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:58.674146 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257aa9a5-8220-439a-8397-5c2e07aaa9ca None admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:58.675030 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257aa9a5-8220-439a-8397-5c2e07aaa9ca None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:58.675494 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257aa9a5-8220-439a-8397-5c2e07aaa9ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:58.677066 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257aa9a5-8220-439a-8397-5c2e07aaa9ca None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:58.681827 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf5820d-81c0-45f7-aa9f-c316689ff5cc None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:58.682123 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf5820d-81c0-45f7-aa9f-c316689ff5cc None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:58.682390 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf5820d-81c0-45f7-aa9f-c316689ff5cc None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:58.684131 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 485/973] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:08:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 17:08:58.686773 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 489/974] 10.222.0.18 () {56 vars in 1115 bytes} [Wed Aug 24 17:08:58 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:08:58.690314 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0907912-a4ad-44dd-a696-c4510a03eaa9 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:58.691325 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0907912-a4ad-44dd-a696-c4510a03eaa9 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:58.692078 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0907912-a4ad-44dd-a696-c4510a03eaa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:58.695787 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257aa9a5-8220-439a-8397-5c2e07aaa9ca None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:58.716183 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-f0907912-a4ad-44dd-a696-c4510a03eaa9 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:08:58.717865 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-f0907912-a4ad-44dd-a696-c4510a03eaa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:08:58.729916 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:58.745694 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:58.794034 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 486/975] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:08:58 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:08:58.799876 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0907912-a4ad-44dd-a696-c4510a03eaa9 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:58.813656 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257aa9a5-8220-439a-8397-5c2e07aaa9ca None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:58.851112 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45ead89-3a28-4706-9afc-a48242aa17a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:58.852103 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45ead89-3a28-4706-9afc-a48242aa17a5 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:58.852595 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45ead89-3a28-4706-9afc-a48242aa17a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:58.853153 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45ead89-3a28-4706-9afc-a48242aa17a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:58.864162 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45ead89-3a28-4706-9afc-a48242aa17a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:58.866306 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45ead89-3a28-4706-9afc-a48242aa17a5 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:58.880840 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-c45ead89-3a28-4706-9afc-a48242aa17a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:08:58.880840 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:58.880840 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:58.880840 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:58.880840 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:58.880840 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:58.880840 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:58.880840 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:58.880840 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:58.880840 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:58.880840 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:58.880840 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:58.880840 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:08:58.880840 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:08:58.880840 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:08:58.880840 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:08:58.881804 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:58.881804 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:58.881804 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:58.881804 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:58.881804 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:58.881804 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:58.881804 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:58.881804 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:58.881804 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:58.881804 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:58.881804 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:58.881804 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:58.881804 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:58.881804 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:58.881804 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:58.881804 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:58.881804 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:08:58.882698 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:08:58.882698 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:08:58.882698 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:08:58.884880 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 490/976] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 24 17:08:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:58.896334 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45ead89-3a28-4706-9afc-a48242aa17a5 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:58.922170 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0907912-a4ad-44dd-a696-c4510a03eaa9 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:58.962855 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef748c1b-0140-4e2a-8df4-d2b0c549770f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:58.963853 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef748c1b-0140-4e2a-8df4-d2b0c549770f None admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:58.964949 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef748c1b-0140-4e2a-8df4-d2b0c549770f None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:58.965514 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef748c1b-0140-4e2a-8df4-d2b0c549770f None admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:58.966335 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef748c1b-0140-4e2a-8df4-d2b0c549770f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:58.968266 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef748c1b-0140-4e2a-8df4-d2b0c549770f None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:58.971094 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: WARNING keystone.server.flask.common [None req-ef748c1b-0140-4e2a-8df4-d2b0c549770f None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 24 17:08:59.001570 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:59.011063 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45ead89-3a28-4706-9afc-a48242aa17a5 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:59.021290 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.sql.core [None req-ef748c1b-0140-4e2a-8df4-d2b0c549770f None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 24 17:08:59.021290 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 24 17:08:59.021290 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [parameters: {'user_id': 'ecab0efc67c146c9820e73b90321739a', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 24 17:08:59.021290 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81732) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 24 17:08:59.030255 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-ef748c1b-0140-4e2a-8df4-d2b0c549770f None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 24 17:08:59.030255 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:59.030255 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 17:08:59.030255 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 17:08:59.030255 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 17:08:59.030255 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 17:08:59.030255 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 17:08:59.030255 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 17:08:59.030255 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 17:08:59.030255 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application conn.query(q) Aug 24 17:08:59.030255 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 17:08:59.030255 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 17:08:59.030255 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 17:08:59.030255 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application result.read() Aug 24 17:08:59.030255 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 17:08:59.037861 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 17:08:59.037861 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 17:08:59.037861 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 17:08:59.037861 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 17:08:59.037861 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 17:08:59.037861 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 17:08:59.037861 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 17:08:59.037861 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 24 17:08:59.037861 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:08:59.037861 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 24 17:08:59.037861 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:08:59.037861 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:59.037861 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 24 17:08:59.037861 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 24 17:08:59.037861 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 24 17:08:59.037861 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 24 17:08:59.037861 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 17:08:59.037861 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application next(self.gen) Aug 24 17:08:59.038788 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 24 17:08:59.038788 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application yield resource Aug 24 17:08:59.038788 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 17:08:59.038788 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application next(self.gen) Aug 24 17:08:59.038788 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 24 17:08:59.038788 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application self.session.rollback() Aug 24 17:08:59.038788 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 24 17:08:59.038788 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application self.force_reraise() Aug 24 17:08:59.038788 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 24 17:08:59.038788 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise self.value Aug 24 17:08:59.038788 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 24 17:08:59.038788 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 24 17:08:59.038788 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 24 17:08:59.038788 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application session.commit() Aug 24 17:08:59.038788 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 24 17:08:59.038788 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 24 17:08:59.038788 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 24 17:08:59.039734 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application self._prepare_impl() Aug 24 17:08:59.039734 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 24 17:08:59.039734 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application self.session.flush() Aug 24 17:08:59.039734 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 24 17:08:59.039734 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application self._flush(objects) Aug 24 17:08:59.039734 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 24 17:08:59.039734 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 24 17:08:59.039734 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 24 17:08:59.039734 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application compat.raise_( Aug 24 17:08:59.039734 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 17:08:59.039734 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception Aug 24 17:08:59.039734 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 24 17:08:59.039734 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application flush_context.execute() Aug 24 17:08:59.039734 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 24 17:08:59.039734 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rec.execute(self) Aug 24 17:08:59.039734 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 24 17:08:59.039734 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 24 17:08:59.041613 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 24 17:08:59.041613 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 24 17:08:59.041613 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 24 17:08:59.041613 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 24 17:08:59.041613 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 24 17:08:59.041613 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 24 17:08:59.041613 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 24 17:08:59.041613 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 24 17:08:59.041613 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 24 17:08:59.041613 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 24 17:08:59.041613 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 24 17:08:59.041613 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 24 17:08:59.041613 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 24 17:08:59.041613 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 24 17:08:59.041613 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 17:08:59.041613 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception Aug 24 17:08:59.042960 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 17:08:59.042960 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 17:08:59.042960 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 17:08:59.042960 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 17:08:59.042960 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 17:08:59.042960 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 17:08:59.042960 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 17:08:59.042960 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application conn.query(q) Aug 24 17:08:59.042960 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 17:08:59.042960 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 17:08:59.042960 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 17:08:59.042960 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application result.read() Aug 24 17:08:59.042960 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 17:08:59.042960 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 17:08:59.042960 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 17:08:59.042960 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 17:08:59.045682 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 17:08:59.045682 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 17:08:59.045682 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 17:08:59.045682 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 17:08:59.045682 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 24 17:08:59.045682 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 24 17:08:59.045682 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [parameters: {'user_id': 'ecab0efc67c146c9820e73b90321739a', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 24 17:08:59.045682 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 24 17:08:59.045682 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:08:59.045682 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 17:08:59.045682 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:08:59.045682 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:59.045682 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:59.045682 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:59.045682 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:59.049298 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:59.049298 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:59.049298 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:59.049298 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:59.049298 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:59.049298 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:59.049298 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:59.049298 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 24 17:08:59.049298 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 24 17:08:59.049298 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:59.049298 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:59.049298 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:08:59.049298 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:08:59.049298 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:08:59.049298 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:08:59.049298 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 24 17:08:59.049298 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 24 17:08:59.051619 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 24 17:08:59.051619 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 24 17:08:59.051619 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 24 17:08:59.051619 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 24 17:08:59.051619 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 24 17:08:59.051619 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:08:59.051619 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 487/977] 10.222.0.18 () {60 vars in 1131 bytes} [Wed Aug 24 17:08:58 2022] POST /identity/v3/users => generated 160 bytes in 241 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:59.051619 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef748c1b-0140-4e2a-8df4-d2b0c549770f None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:59.057837 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:59.062341 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c237b8a4-e230-4e63-bcd0-85c8b6ab739c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:59.063162 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c237b8a4-e230-4e63-bcd0-85c8b6ab739c None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:59.063440 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c237b8a4-e230-4e63-bcd0-85c8b6ab739c None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:59.063714 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c237b8a4-e230-4e63-bcd0-85c8b6ab739c None admin] PATH_INFO: `/v3/domains` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:59.068838 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c237b8a4-e230-4e63-bcd0-85c8b6ab739c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:59.068838 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c237b8a4-e230-4e63-bcd0-85c8b6ab739c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:59.068838 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c237b8a4-e230-4e63-bcd0-85c8b6ab739c None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:59.079104 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 491/978] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 24 17:08:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:59.079700 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c237b8a4-e230-4e63-bcd0-85c8b6ab739c None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:59.167769 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef748c1b-0140-4e2a-8df4-d2b0c549770f None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:59.198496 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c237b8a4-e230-4e63-bcd0-85c8b6ab739c None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:59.216638 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1828ae7b-23e4-465d-a997-ee8583ca90ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:59.217588 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1828ae7b-23e4-465d-a997-ee8583ca90ca None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:59.217875 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1828ae7b-23e4-465d-a997-ee8583ca90ca None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:59.218375 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1828ae7b-23e4-465d-a997-ee8583ca90ca None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:59.227558 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1828ae7b-23e4-465d-a997-ee8583ca90ca None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:59.229713 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1828ae7b-23e4-465d-a997-ee8583ca90ca None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:59.237135 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3a0b79-6f27-4d39-9b39-be89f6cc523c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:59.237745 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3a0b79-6f27-4d39-9b39-be89f6cc523c None admin] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:59.238025 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3a0b79-6f27-4d39-9b39-be89f6cc523c None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:59.238297 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3a0b79-6f27-4d39-9b39-be89f6cc523c None admin] PATH_INFO: `/v3/services` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:59.238964 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3a0b79-6f27-4d39-9b39-be89f6cc523c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:59.240308 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3a0b79-6f27-4d39-9b39-be89f6cc523c None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:59.241469 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-1828ae7b-23e4-465d-a997-ee8583ca90ca None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 17:08:59.241469 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:59.241469 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:59.241469 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:59.241469 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:59.241469 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:59.241469 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:59.241469 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:59.241469 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:59.241469 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:59.241469 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:59.241469 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:59.241469 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:08:59.241469 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:08:59.241469 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:08:59.241469 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:08:59.242435 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:59.242435 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:59.242435 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:08:59.242435 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:08:59.242435 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:08:59.242435 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:08:59.242435 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:59.242435 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:59.242435 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:59.242435 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:59.242435 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:59.242435 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:59.242435 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:59.242435 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:59.242435 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:59.242435 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:59.242435 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:59.242435 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:59.243810 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:59.243810 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:59.243810 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:08:59.243810 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:08:59.243810 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:08:59.243810 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:08:59.243810 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:08:59.243810 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:08:59.243810 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 17:08:59.243810 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:08:59.247042 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 488/979] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 24 17:08:59 2022] GET /identity/v3/users/cinder => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:59.247867 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1828ae7b-23e4-465d-a997-ee8583ca90ca None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:59.314115 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:59.350954 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 492/980] 10.222.0.18 () {60 vars in 1141 bytes} [Wed Aug 24 17:08:59 2022] POST /identity/v3/services => generated 242 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:08:59.350954 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3a0b79-6f27-4d39-9b39-be89f6cc523c None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:59.356540 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1828ae7b-23e4-465d-a997-ee8583ca90ca None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:59.366515 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:59.423098 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b46ba09-557c-4d6f-aeb4-e886366ad9a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:59.424058 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b46ba09-557c-4d6f-aeb4-e886366ad9a9 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:59.424348 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b46ba09-557c-4d6f-aeb4-e886366ad9a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:59.424624 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b46ba09-557c-4d6f-aeb4-e886366ad9a9 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:59.436575 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b46ba09-557c-4d6f-aeb4-e886366ad9a9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:59.438434 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b46ba09-557c-4d6f-aeb4-e886366ad9a9 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:59.456155 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-8b46ba09-557c-4d6f-aeb4-e886366ad9a9 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 17:08:59.456155 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:08:59.456155 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:08:59.456155 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:08:59.456155 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:08:59.456155 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:08:59.456155 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:08:59.456155 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:08:59.456155 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:08:59.456155 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:08:59.456155 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:08:59.456155 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:08:59.456155 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:08:59.456155 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:08:59.456155 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:08:59.456155 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:08:59.457755 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:08:59.457755 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:08:59.457755 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:08:59.457755 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:08:59.457755 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:08:59.457755 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:08:59.457755 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:08:59.457755 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:08:59.457755 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:08:59.457755 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:08:59.457755 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:08:59.457755 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 24 17:08:59.457755 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:08:59.457755 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:08:59.457755 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:08:59.457755 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:08:59.457755 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:08:59.457755 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:08:59.458844 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:08:59.458844 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:08:59.458844 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:08:59.458844 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:08:59.458844 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:08:59.458844 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:08:59.458844 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:08:59.458844 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:08:59.458844 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 17:08:59.458844 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:08:59.462446 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 489/981] 10.222.0.18 () {56 vars in 1117 bytes} [Wed Aug 24 17:08:59 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:08:59.465033 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b46ba09-557c-4d6f-aeb4-e886366ad9a9 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:59.489638 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3a0b79-6f27-4d39-9b39-be89f6cc523c None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:59.538940 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72ba19c-eee0-4f26-971e-80e1fb9d1e0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:59.539740 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72ba19c-eee0-4f26-971e-80e1fb9d1e0d None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:59.540026 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72ba19c-eee0-4f26-971e-80e1fb9d1e0d None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:59.540304 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72ba19c-eee0-4f26-971e-80e1fb9d1e0d None admin] PATH_INFO: `/v3/users` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:59.541012 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72ba19c-eee0-4f26-971e-80e1fb9d1e0d None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:59.541316 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72ba19c-eee0-4f26-971e-80e1fb9d1e0d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:59.542829 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72ba19c-eee0-4f26-971e-80e1fb9d1e0d None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:59.586576 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 493/982] 10.222.0.18 () {56 vars in 1101 bytes} [Wed Aug 24 17:08:59 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:08:59.601161 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b46ba09-557c-4d6f-aeb4-e886366ad9a9 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:59.629924 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:59.643775 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c49886-2835-4801-9c2b-46e2f2c10d14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:59.648529 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c49886-2835-4801-9c2b-46e2f2c10d14 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:59.648529 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c49886-2835-4801-9c2b-46e2f2c10d14 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:59.648529 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c49886-2835-4801-9c2b-46e2f2c10d14 None admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:59.648529 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c49886-2835-4801-9c2b-46e2f2c10d14 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:08:59.648529 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c49886-2835-4801-9c2b-46e2f2c10d14 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:59.648529 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c49886-2835-4801-9c2b-46e2f2c10d14 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:59.669622 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 490/983] 10.222.0.18 () {56 vars in 1149 bytes} [Wed Aug 24 17:08:59 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:08:59.676059 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72ba19c-eee0-4f26-971e-80e1fb9d1e0d None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:08:59.762531 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:08:59.797417 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72ba19c-eee0-4f26-971e-80e1fb9d1e0d None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:08:59.835524 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b97ab59-4028-4c35-8f8a-1dbcebe6fdc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:08:59.836556 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b97ab59-4028-4c35-8f8a-1dbcebe6fdc0 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:08:59.837106 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b97ab59-4028-4c35-8f8a-1dbcebe6fdc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:08:59.837630 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b97ab59-4028-4c35-8f8a-1dbcebe6fdc0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:08:59.909058 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b97ab59-4028-4c35-8f8a-1dbcebe6fdc0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:08:59.911282 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b97ab59-4028-4c35-8f8a-1dbcebe6fdc0 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:08:59.975737 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:09:00.022513 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 494/984] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 24 17:08:59 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 17:09:00.024254 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c49886-2835-4801-9c2b-46e2f2c10d14 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:00.125873 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c49886-2835-4801-9c2b-46e2f2c10d14 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:00.161198 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e225e4-14bf-41cb-8231-ac23f61bd6c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:00.162173 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e225e4-14bf-41cb-8231-ac23f61bd6c1 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:00.162683 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e225e4-14bf-41cb-8231-ac23f61bd6c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:00.163193 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e225e4-14bf-41cb-8231-ac23f61bd6c1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:00.174237 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e225e4-14bf-41cb-8231-ac23f61bd6c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:00.175980 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e225e4-14bf-41cb-8231-ac23f61bd6c1 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:00.191299 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-70e225e4-14bf-41cb-8231-ac23f61bd6c1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:09:00.191299 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:09:00.191299 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:09:00.191299 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:09:00.191299 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:09:00.191299 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:09:00.191299 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:09:00.191299 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:09:00.191299 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:09:00.191299 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:09:00.191299 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:09:00.191299 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:09:00.191299 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:09:00.191299 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:09:00.191299 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:09:00.191299 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:09:00.192282 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:09:00.192282 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:09:00.192282 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:09:00.192282 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:09:00.192282 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:09:00.192282 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:09:00.192282 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:09:00.192282 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 24 17:09:00.192282 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:09:00.192282 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:09:00.192282 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:09:00.192282 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:09:00.192282 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:09:00.192282 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:09:00.192282 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:09:00.192282 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:09:00.192282 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:09:00.193209 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:09:00.193209 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:09:00.193209 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:09:00.199761 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 491/985] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 24 17:09:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:09:00.200632 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b97ab59-4028-4c35-8f8a-1dbcebe6fdc0 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:00.303513 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b97ab59-4028-4c35-8f8a-1dbcebe6fdc0 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:00.337506 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc27e89-38d7-4125-993b-6ce45841540c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:00.338432 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc27e89-38d7-4125-993b-6ce45841540c None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:00.338983 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc27e89-38d7-4125-993b-6ce45841540c None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:00.339500 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc27e89-38d7-4125-993b-6ce45841540c None admin] PATH_INFO: `/v3/domains` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:00.340336 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc27e89-38d7-4125-993b-6ce45841540c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:09:00.340867 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc27e89-38d7-4125-993b-6ce45841540c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:00.342425 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc27e89-38d7-4125-993b-6ce45841540c None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:00.357883 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 495/986] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 24 17:09:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:09:00.360430 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e225e4-14bf-41cb-8231-ac23f61bd6c1 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:00.397091 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:09:00.463876 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e225e4-14bf-41cb-8231-ac23f61bd6c1 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:00.499722 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adccf161-9b38-43a5-ba87-55d93f7b60dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:00.500357 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adccf161-9b38-43a5-ba87-55d93f7b60dc None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:00.500636 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adccf161-9b38-43a5-ba87-55d93f7b60dc None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:00.500906 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adccf161-9b38-43a5-ba87-55d93f7b60dc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:00.507911 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adccf161-9b38-43a5-ba87-55d93f7b60dc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:00.509339 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adccf161-9b38-43a5-ba87-55d93f7b60dc None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:00.521083 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-adccf161-9b38-43a5-ba87-55d93f7b60dc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:09:00.521083 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:09:00.521083 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:09:00.521083 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:09:00.521083 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:09:00.521083 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:09:00.521083 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:09:00.521083 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:09:00.521083 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:09:00.521083 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:09:00.521083 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:09:00.521083 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:09:00.521083 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:09:00.521083 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:09:00.521083 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:09:00.522152 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:09:00.522152 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:09:00.522152 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:09:00.522152 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:09:00.522152 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:09:00.522152 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:09:00.522152 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:09:00.522152 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:09:00.522152 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 24 17:09:00.522152 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:09:00.522152 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:09:00.522152 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:09:00.522152 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:09:00.522152 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:09:00.522152 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:09:00.522152 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:09:00.522152 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:09:00.523095 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:09:00.523095 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:09:00.523095 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:09:00.523095 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:09:00.523095 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:09:00.523095 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:09:00.523095 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:09:00.523095 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:09:00.523095 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 492/987] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 24 17:09:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:09:00.530047 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc27e89-38d7-4125-993b-6ce45841540c None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:00.562265 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:09:00.610756 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178d564f-ffab-4549-871d-7817ad25f150 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:00.611108 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178d564f-ffab-4549-871d-7817ad25f150 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:00.611345 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178d564f-ffab-4549-871d-7817ad25f150 None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:00.612831 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 493/988] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:09:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 17:09:00.617604 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f1ad58-fc87-4d87-9c49-1a532ea8f77b None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:00.617882 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f1ad58-fc87-4d87-9c49-1a532ea8f77b None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:00.618176 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f1ad58-fc87-4d87-9c49-1a532ea8f77b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:00.637446 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-a5f1ad58-fc87-4d87-9c49-1a532ea8f77b None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:09:00.638982 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-a5f1ad58-fc87-4d87-9c49-1a532ea8f77b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:09:00.640549 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc27e89-38d7-4125-993b-6ce45841540c None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:00.675308 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad80309-f975-4630-93a5-6b2c5de07283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:00.676181 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad80309-f975-4630-93a5-6b2c5de07283 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:00.676646 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad80309-f975-4630-93a5-6b2c5de07283 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:00.677109 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad80309-f975-4630-93a5-6b2c5de07283 None admin] PATH_INFO: `/v3/projects` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:00.677973 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad80309-f975-4630-93a5-6b2c5de07283 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:09:00.678491 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad80309-f975-4630-93a5-6b2c5de07283 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:00.680181 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad80309-f975-4630-93a5-6b2c5de07283 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:00.698243 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 496/989] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Aug 24 17:09:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:09:00.700242 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad80309-f975-4630-93a5-6b2c5de07283 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:00.707364 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 494/990] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:09:00 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:09:00.792496 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:09:00.804911 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad80309-f975-4630-93a5-6b2c5de07283 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:00.839681 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9917e418-64e0-4a53-b589-f0ffcb647b4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:00.840288 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9917e418-64e0-4a53-b589-f0ffcb647b4e None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:00.840628 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9917e418-64e0-4a53-b589-f0ffcb647b4e None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:00.840906 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9917e418-64e0-4a53-b589-f0ffcb647b4e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:00.841628 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9917e418-64e0-4a53-b589-f0ffcb647b4e None admin] RBAC: Adding query filter params (role.id=967c5aaa9ba746a7acaa59f3e9691b31, scope.project.id=46ab158bc3644c888557781b0630b0b8, user.id=0ccc6723f7514e52863d102de3839951) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:09:00.841937 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9917e418-64e0-4a53-b589-f0ffcb647b4e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:00.843395 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9917e418-64e0-4a53-b589-f0ffcb647b4e None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:00.855125 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 497/991] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 24 17:09:00 2022] GET /identity/v3/role_assignments?user.id=0ccc6723f7514e52863d102de3839951&scope.project.id=46ab158bc3644c888557781b0630b0b8&role.id=967c5aaa9ba746a7acaa59f3e9691b31 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:09:00.974975 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f1ad58-fc87-4d87-9c49-1a532ea8f77b None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:01.002282 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:09:01.080918 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f1ad58-fc87-4d87-9c49-1a532ea8f77b None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:01.114883 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73aeffb-eefb-48a0-b9c3-6f799df93635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:01.115530 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73aeffb-eefb-48a0-b9c3-6f799df93635 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:01.115807 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73aeffb-eefb-48a0-b9c3-6f799df93635 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:01.116079 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73aeffb-eefb-48a0-b9c3-6f799df93635 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:01.116649 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73aeffb-eefb-48a0-b9c3-6f799df93635 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:01.118005 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73aeffb-eefb-48a0-b9c3-6f799df93635 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:01.127384 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-c73aeffb-eefb-48a0-b9c3-6f799df93635 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 17:09:01.127384 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:09:01.127384 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 17:09:01.127384 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 17:09:01.127384 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 17:09:01.127384 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 17:09:01.127384 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 17:09:01.127384 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:09:01.127384 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 17:09:01.127384 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:09:01.127384 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 17:09:01.127384 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:09:01.127384 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:09:01.127384 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:09:01.127384 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:09:01.127384 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:09:01.127384 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:09:01.128430 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:09:01.128430 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:09:01.128430 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:09:01.128430 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:09:01.128430 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:09:01.128430 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:09:01.128430 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 17:09:01.128430 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 17:09:01.128430 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 17:09:01.128430 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 17:09:01.128430 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:09:01.128430 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:09:01.128430 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:09:01.128430 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:09:01.128430 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:09:01.128430 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:09:01.129952 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:09:01.129952 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 24 17:09:01.129952 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:09:01.129952 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:09:01.129952 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:09:01.129952 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:09:01.129952 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:09:01.129952 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:09:01.129952 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:09:01.129952 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:09:01.129952 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 17:09:01.129952 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:09:01.129952 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 17:09:01.129952 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:09:01.129952 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 495/992] 10.222.0.18 () {56 vars in 1126 bytes} [Wed Aug 24 17:09:00 2022] GET /identity/v3/services/object-store => generated 93 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:09:01.134583 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9917e418-64e0-4a53-b589-f0ffcb647b4e None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:01.237421 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9917e418-64e0-4a53-b589-f0ffcb647b4e None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:01.271047 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbd7eba-33ce-4908-afd0-3dc7907af83b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:01.271649 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbd7eba-33ce-4908-afd0-3dc7907af83b None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:01.271921 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbd7eba-33ce-4908-afd0-3dc7907af83b None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:01.272187 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbd7eba-33ce-4908-afd0-3dc7907af83b None admin] PATH_INFO: `/v3/services` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:01.272807 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbd7eba-33ce-4908-afd0-3dc7907af83b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:09:01.273106 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbd7eba-33ce-4908-afd0-3dc7907af83b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:01.274440 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbd7eba-33ce-4908-afd0-3dc7907af83b None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:01.279906 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 498/993] 10.222.0.18 () {56 vars in 1122 bytes} [Wed Aug 24 17:09:01 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:09:01.285365 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73aeffb-eefb-48a0-b9c3-6f799df93635 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:01.333610 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac51cfa5-de54-4db5-9973-f8c86d529421 None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:01.333970 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac51cfa5-de54-4db5-9973-f8c86d529421 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:01.334174 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac51cfa5-de54-4db5-9973-f8c86d529421 None None] PATH_INFO: `/` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:01.335584 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 499/994] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:09:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 17:09:01.341154 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b38d5b-b6d2-4d57-b621-e0ba57d322d3 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:01.341436 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b38d5b-b6d2-4d57-b621-e0ba57d322d3 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:01.341766 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b38d5b-b6d2-4d57-b621-e0ba57d322d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:01.360254 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-29b38d5b-b6d2-4d57-b621-e0ba57d322d3 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:09:01.365759 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-29b38d5b-b6d2-4d57-b621-e0ba57d322d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:09:01.373041 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:09:01.380319 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:09:01.393049 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73aeffb-eefb-48a0-b9c3-6f799df93635 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:01.429921 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9a4c04-50e3-4005-aa55-117085911ba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:01.430951 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 500/995] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:09:01 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 17:09:01.431103 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9a4c04-50e3-4005-aa55-117085911ba3 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:01.431103 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9a4c04-50e3-4005-aa55-117085911ba3 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:01.431251 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9a4c04-50e3-4005-aa55-117085911ba3 None admin] PATH_INFO: `/v3/services` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:01.431860 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9a4c04-50e3-4005-aa55-117085911ba3 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:09:01.432168 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9a4c04-50e3-4005-aa55-117085911ba3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:01.433683 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9a4c04-50e3-4005-aa55-117085911ba3 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:01.439122 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 496/996] 10.222.0.18 () {56 vars in 1122 bytes} [Wed Aug 24 17:09:01 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:09:01.446582 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b38d5b-b6d2-4d57-b621-e0ba57d322d3 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:01.553533 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b38d5b-b6d2-4d57-b621-e0ba57d322d3 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:01.593229 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75004aa5-70ef-4f29-ae5b-aaa65301bb61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:01.593840 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75004aa5-70ef-4f29-ae5b-aaa65301bb61 None admin] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:01.594120 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75004aa5-70ef-4f29-ae5b-aaa65301bb61 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:01.594393 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75004aa5-70ef-4f29-ae5b-aaa65301bb61 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:01.595082 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75004aa5-70ef-4f29-ae5b-aaa65301bb61 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:01.596436 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75004aa5-70ef-4f29-ae5b-aaa65301bb61 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:01.662936 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 501/997] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 24 17:09:01 2022] POST /identity/v3/endpoints => generated 325 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:09:01.712966 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9a4c04-50e3-4005-aa55-117085911ba3 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:01.823655 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9a4c04-50e3-4005-aa55-117085911ba3 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:01.863981 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c6882f-a550-4c4f-81b6-298f0a38ba34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:01.864874 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c6882f-a550-4c4f-81b6-298f0a38ba34 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:01.865307 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c6882f-a550-4c4f-81b6-298f0a38ba34 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:01.865722 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c6882f-a550-4c4f-81b6-298f0a38ba34 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:01.866454 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c6882f-a550-4c4f-81b6-298f0a38ba34 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:01.868012 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c6882f-a550-4c4f-81b6-298f0a38ba34 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:01.877709 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-44c6882f-a550-4c4f-81b6-298f0a38ba34 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 17:09:01.877709 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:09:01.877709 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 17:09:01.877709 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 17:09:01.877709 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 17:09:01.877709 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 17:09:01.877709 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 17:09:01.877709 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:09:01.877709 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 17:09:01.877709 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:09:01.877709 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 17:09:01.877709 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:09:01.877709 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:09:01.877709 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:09:01.877709 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:09:01.877709 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:09:01.877709 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:09:01.878882 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:09:01.878882 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:09:01.878882 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:09:01.878882 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:09:01.878882 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:09:01.878882 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:09:01.878882 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 17:09:01.878882 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 17:09:01.878882 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 17:09:01.878882 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 17:09:01.878882 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:09:01.878882 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:09:01.878882 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:09:01.878882 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:09:01.878882 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:09:01.878882 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:09:01.879782 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:09:01.879782 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 24 17:09:01.879782 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:09:01.879782 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:09:01.879782 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:09:01.879782 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:09:01.879782 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:09:01.879782 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:09:01.879782 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:09:01.879782 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:09:01.879782 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 17:09:01.879782 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:09:01.879782 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 17:09:01.879782 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:09:01.880539 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 497/998] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 24 17:09:01 2022] GET /identity/v3/services/compute => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:09:01.886356 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75004aa5-70ef-4f29-ae5b-aaa65301bb61 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:01.895901 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:09:01.990019 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75004aa5-70ef-4f29-ae5b-aaa65301bb61 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:02.023910 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e378bd3-2271-4284-9262-5a3e0079796b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:02.024519 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e378bd3-2271-4284-9262-5a3e0079796b None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:02.024798 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e378bd3-2271-4284-9262-5a3e0079796b None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:02.025071 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e378bd3-2271-4284-9262-5a3e0079796b None admin] PATH_INFO: `/v3/services` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:02.025701 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e378bd3-2271-4284-9262-5a3e0079796b None admin] RBAC: Adding query filter params (name=compute) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:09:02.026004 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e378bd3-2271-4284-9262-5a3e0079796b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:02.027913 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e378bd3-2271-4284-9262-5a3e0079796b None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:02.036899 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 502/999] 10.222.0.18 () {56 vars in 1112 bytes} [Wed Aug 24 17:09:01 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:09:02.043114 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c6882f-a550-4c4f-81b6-298f0a38ba34 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:02.058786 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:09:02.146288 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c6882f-a550-4c4f-81b6-298f0a38ba34 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:02.181427 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91427aae-6a70-4cfe-8521-e66786c926ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:02.182031 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91427aae-6a70-4cfe-8521-e66786c926ba None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:02.182309 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91427aae-6a70-4cfe-8521-e66786c926ba None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:02.182581 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91427aae-6a70-4cfe-8521-e66786c926ba None admin] PATH_INFO: `/v3/services` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:02.183259 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91427aae-6a70-4cfe-8521-e66786c926ba None admin] RBAC: Adding query filter params (type=compute) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:09:02.183561 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91427aae-6a70-4cfe-8521-e66786c926ba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:02.184919 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91427aae-6a70-4cfe-8521-e66786c926ba None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:02.194614 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 498/1000] 10.222.0.18 () {56 vars in 1112 bytes} [Wed Aug 24 17:09:02 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:09:02.195596 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e378bd3-2271-4284-9262-5a3e0079796b None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:02.265604 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:09:02.296764 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e378bd3-2271-4284-9262-5a3e0079796b None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:02.329908 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5899aa3-c861-4560-9e2c-2445f17b2b53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:02.330520 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5899aa3-c861-4560-9e2c-2445f17b2b53 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:02.330799 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5899aa3-c861-4560-9e2c-2445f17b2b53 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:02.331099 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5899aa3-c861-4560-9e2c-2445f17b2b53 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:02.331910 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5899aa3-c861-4560-9e2c-2445f17b2b53 None admin] RBAC: Adding query filter params (interface=public, service_id=02387a7b5c2845fab5fa7addd7f95c81, region_id=RegionOne) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:09:02.332218 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5899aa3-c861-4560-9e2c-2445f17b2b53 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:02.333673 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5899aa3-c861-4560-9e2c-2445f17b2b53 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:02.343217 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 503/1001] 10.222.0.18 () {56 vars in 1251 bytes} [Wed Aug 24 17:09:02 2022] GET /identity/v3/endpoints?service_id=02387a7b5c2845fab5fa7addd7f95c81&interface=public®ion_id=RegionOne => generated 194 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:09:02.345278 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91427aae-6a70-4cfe-8521-e66786c926ba None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:02.426810 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:09:02.449348 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91427aae-6a70-4cfe-8521-e66786c926ba None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:02.483402 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64985c52-caaf-485a-8111-b145ba35aecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:02.484006 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64985c52-caaf-485a-8111-b145ba35aecf None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:02.484285 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64985c52-caaf-485a-8111-b145ba35aecf None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:02.484556 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64985c52-caaf-485a-8111-b145ba35aecf None admin] PATH_INFO: `/v3/services` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:02.485151 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64985c52-caaf-485a-8111-b145ba35aecf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:02.486511 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64985c52-caaf-485a-8111-b145ba35aecf None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:02.499179 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 499/1002] 10.222.0.18 () {56 vars in 1087 bytes} [Wed Aug 24 17:09:02 2022] GET /identity/v3/services => generated 1727 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:09:02.809692 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110eb34d-f1a0-437e-9f47-3ce170cd55a2 None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:02.809883 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110eb34d-f1a0-437e-9f47-3ce170cd55a2 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:02.810174 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110eb34d-f1a0-437e-9f47-3ce170cd55a2 None None] PATH_INFO: `/` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:02.811677 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 504/1003] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:09:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:09:02.816643 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2859974-b22c-4e3a-9550-8e082c34099e None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:02.816934 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2859974-b22c-4e3a-9550-8e082c34099e None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:02.817210 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2859974-b22c-4e3a-9550-8e082c34099e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:02.835616 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-e2859974-b22c-4e3a-9550-8e082c34099e None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:09:02.836796 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-e2859974-b22c-4e3a-9550-8e082c34099e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:09:02.907001 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 500/1004] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:09:02 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:09:03.168077 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110eb34d-f1a0-437e-9f47-3ce170cd55a2 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:03.270451 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110eb34d-f1a0-437e-9f47-3ce170cd55a2 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:03.306159 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730a9ec8-cb5e-4981-9384-c44d0d7dd318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:03.306792 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730a9ec8-cb5e-4981-9384-c44d0d7dd318 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:03.307096 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730a9ec8-cb5e-4981-9384-c44d0d7dd318 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:03.307367 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730a9ec8-cb5e-4981-9384-c44d0d7dd318 None admin] PATH_INFO: `/v3/services/image` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:03.307994 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730a9ec8-cb5e-4981-9384-c44d0d7dd318 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:03.309344 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730a9ec8-cb5e-4981-9384-c44d0d7dd318 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:03.311130 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:09:03.318782 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-730a9ec8-cb5e-4981-9384-c44d0d7dd318 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 17:09:03.318782 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:09:03.318782 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 17:09:03.318782 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 17:09:03.318782 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 17:09:03.318782 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 17:09:03.318782 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 17:09:03.318782 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:09:03.318782 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 17:09:03.318782 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:09:03.318782 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 17:09:03.318782 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:09:03.318782 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:09:03.318782 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:09:03.318782 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:09:03.318782 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:09:03.318782 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:09:03.319795 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:09:03.319795 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:09:03.319795 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:09:03.319795 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:09:03.319795 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:09:03.319795 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:09:03.319795 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 17:09:03.319795 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 17:09:03.319795 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 17:09:03.319795 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 17:09:03.319795 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:09:03.319795 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:09:03.319795 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:09:03.319795 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:09:03.319795 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:09:03.319795 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:09:03.320637 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:09:03.320637 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application with Lock( Aug 24 17:09:03.320637 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:09:03.320637 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:09:03.320637 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:09:03.320637 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:09:03.320637 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:09:03.320637 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:09:03.320637 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:09:03.320637 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:09:03.320637 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 17:09:03.320637 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:09:03.320637 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 17:09:03.320637 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:09:03.322983 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 505/1005] 10.222.0.18 () {56 vars in 1105 bytes} [Wed Aug 24 17:09:03 2022] GET /identity/v3/services/image => generated 86 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:09:03.326396 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2859974-b22c-4e3a-9550-8e082c34099e None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:03.398812 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:09:03.429151 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2859974-b22c-4e3a-9550-8e082c34099e None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:03.463923 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6e9fc4-b8f4-4d53-ba1e-c7fd6471a134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:03.464610 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6e9fc4-b8f4-4d53-ba1e-c7fd6471a134 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:03.464881 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6e9fc4-b8f4-4d53-ba1e-c7fd6471a134 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:03.465153 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6e9fc4-b8f4-4d53-ba1e-c7fd6471a134 None admin] PATH_INFO: `/v3/services` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:03.465775 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6e9fc4-b8f4-4d53-ba1e-c7fd6471a134 None admin] RBAC: Adding query filter params (name=image) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:09:03.466073 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6e9fc4-b8f4-4d53-ba1e-c7fd6471a134 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:03.467476 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6e9fc4-b8f4-4d53-ba1e-c7fd6471a134 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:03.476961 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 501/1006] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 24 17:09:03 2022] GET /identity/v3/services?name=image => generated 122 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:09:03.478536 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730a9ec8-cb5e-4981-9384-c44d0d7dd318 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:03.590107 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730a9ec8-cb5e-4981-9384-c44d0d7dd318 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:03.615809 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2529a3d-46a7-481c-9386-3e15adde1ac6 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:03.616106 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2529a3d-46a7-481c-9386-3e15adde1ac6 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:03.616363 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2529a3d-46a7-481c-9386-3e15adde1ac6 None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:03.617719 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 502/1007] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:09:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:09:03.622806 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58b95c0-de31-46ae-b080-7a02fd64c6a0 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:03.623140 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58b95c0-de31-46ae-b080-7a02fd64c6a0 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:03.623420 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58b95c0-de31-46ae-b080-7a02fd64c6a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:03.625745 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd62ec94-0b46-4bc1-a7b2-0508c13f6102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:03.626632 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd62ec94-0b46-4bc1-a7b2-0508c13f6102 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:03.626894 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd62ec94-0b46-4bc1-a7b2-0508c13f6102 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:03.627214 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd62ec94-0b46-4bc1-a7b2-0508c13f6102 None admin] PATH_INFO: `/v3/services` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:03.627828 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd62ec94-0b46-4bc1-a7b2-0508c13f6102 None admin] RBAC: Adding query filter params (type=image) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:09:03.628130 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd62ec94-0b46-4bc1-a7b2-0508c13f6102 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:03.629484 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd62ec94-0b46-4bc1-a7b2-0508c13f6102 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:03.643151 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 506/1008] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 24 17:09:03 2022] GET /identity/v3/services?type=image => generated 122 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:09:03.643377 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-f58b95c0-de31-46ae-b080-7a02fd64c6a0 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:09:03.643377 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-f58b95c0-de31-46ae-b080-7a02fd64c6a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:09:03.723147 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 503/1009] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:09:03 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 17:09:03.994337 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd62ec94-0b46-4bc1-a7b2-0508c13f6102 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:04.056451 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:09:04.097998 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd62ec94-0b46-4bc1-a7b2-0508c13f6102 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:04.132179 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86092f3-a656-4a3a-bc30-3d909526f113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:04.132806 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86092f3-a656-4a3a-bc30-3d909526f113 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:04.133088 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86092f3-a656-4a3a-bc30-3d909526f113 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:04.133361 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86092f3-a656-4a3a-bc30-3d909526f113 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:04.134474 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86092f3-a656-4a3a-bc30-3d909526f113 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:04.135923 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86092f3-a656-4a3a-bc30-3d909526f113 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:04.138185 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 507/1010] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 24 17:09:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:09:04.143737 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58b95c0-de31-46ae-b080-7a02fd64c6a0 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:04.205460 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:09:04.243375 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58b95c0-de31-46ae-b080-7a02fd64c6a0 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:04.278471 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f31bbc-77c0-42bf-9bd1-50b3f4c62b7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:04.279025 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f31bbc-77c0-42bf-9bd1-50b3f4c62b7d None admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:04.279303 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f31bbc-77c0-42bf-9bd1-50b3f4c62b7d None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:04.279573 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f31bbc-77c0-42bf-9bd1-50b3f4c62b7d None admin] PATH_INFO: `/v3/projects` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:04.280207 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f31bbc-77c0-42bf-9bd1-50b3f4c62b7d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:04.281618 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f31bbc-77c0-42bf-9bd1-50b3f4c62b7d None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:04.374926 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 504/1011] 10.222.0.18 () {60 vars in 1141 bytes} [Wed Aug 24 17:09:04 2022] POST /identity/v3/projects => generated 311 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:09:04.403358 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58a1b22-06c4-46d1-9a20-3ca53be691e5 None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:04.403929 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58a1b22-06c4-46d1-9a20-3ca53be691e5 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:04.403929 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58a1b22-06c4-46d1-9a20-3ca53be691e5 None None] PATH_INFO: `/` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:04.405440 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 508/1012] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:09:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:09:04.410991 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07069c82-871c-477f-b372-0b45f10b5764 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:04.411300 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07069c82-871c-477f-b372-0b45f10b5764 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:04.411560 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07069c82-871c-477f-b372-0b45f10b5764 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:04.431108 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-07069c82-871c-477f-b372-0b45f10b5764 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:09:04.432381 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-07069c82-871c-477f-b372-0b45f10b5764 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:09:04.508334 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 505/1013] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:09:04 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:09:04.776330 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58a1b22-06c4-46d1-9a20-3ca53be691e5 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:04.879750 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58a1b22-06c4-46d1-9a20-3ca53be691e5 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:04.908154 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:09:04.916342 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6baa55f-33e9-4685-a1d3-238f80adcc57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:04.916999 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6baa55f-33e9-4685-a1d3-238f80adcc57 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:04.917803 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6baa55f-33e9-4685-a1d3-238f80adcc57 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:04.918076 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6baa55f-33e9-4685-a1d3-238f80adcc57 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:04.918658 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6baa55f-33e9-4685-a1d3-238f80adcc57 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:04.920053 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6baa55f-33e9-4685-a1d3-238f80adcc57 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:04.928715 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-c6baa55f-33e9-4685-a1d3-238f80adcc57 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 17:09:04.928715 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:09:04.928715 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 17:09:04.928715 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 17:09:04.928715 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 17:09:04.928715 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 17:09:04.928715 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 17:09:04.928715 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:09:04.928715 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 17:09:04.928715 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:09:04.928715 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 17:09:04.928715 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:09:04.928715 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:09:04.928715 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:09:04.928715 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:09:04.928715 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:09:04.928715 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:09:04.931694 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:09:04.931694 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:09:04.931694 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:09:04.931694 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:09:04.931694 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:09:04.931694 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:09:04.931694 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 17:09:04.931694 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 17:09:04.931694 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 17:09:04.931694 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 17:09:04.931694 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:09:04.931694 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:09:04.931694 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:09:04.931694 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:09:04.931694 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:09:04.931694 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:09:04.932607 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:09:04.932607 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application with Lock( Aug 24 17:09:04.932607 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:09:04.932607 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:09:04.932607 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:09:04.932607 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:09:04.932607 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:09:04.932607 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:09:04.932607 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:09:04.932607 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:09:04.932607 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 17:09:04.932607 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:09:04.932607 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 17:09:04.932607 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:09:04.932607 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 509/1014] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 24 17:09:04 2022] GET /identity/v3/services/compute => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:09:04.936151 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07069c82-871c-477f-b372-0b45f10b5764 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:04.982595 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:09:05.037367 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07069c82-871c-477f-b372-0b45f10b5764 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:05.072313 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993e0618-271d-471a-862e-a754ac901258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:05.072923 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993e0618-271d-471a-862e-a754ac901258 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:05.073203 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993e0618-271d-471a-862e-a754ac901258 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:05.073503 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993e0618-271d-471a-862e-a754ac901258 None admin] PATH_INFO: `/v3/services` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:05.074135 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993e0618-271d-471a-862e-a754ac901258 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:09:05.074436 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993e0618-271d-471a-862e-a754ac901258 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:05.075820 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993e0618-271d-471a-862e-a754ac901258 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:05.085112 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 506/1015] 10.222.0.18 () {56 vars in 1112 bytes} [Wed Aug 24 17:09:04 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:09:05.086660 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6baa55f-33e9-4685-a1d3-238f80adcc57 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:05.185485 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6baa55f-33e9-4685-a1d3-238f80adcc57 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:05.223336 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf46ba4c-1bbe-4e11-8fa4-c5cabbd504d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:05.224002 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf46ba4c-1bbe-4e11-8fa4-c5cabbd504d2 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:05.224283 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf46ba4c-1bbe-4e11-8fa4-c5cabbd504d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:05.224578 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf46ba4c-1bbe-4e11-8fa4-c5cabbd504d2 None admin] PATH_INFO: `/v3/services` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:05.225210 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf46ba4c-1bbe-4e11-8fa4-c5cabbd504d2 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:09:05.225513 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf46ba4c-1bbe-4e11-8fa4-c5cabbd504d2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:05.226865 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf46ba4c-1bbe-4e11-8fa4-c5cabbd504d2 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:05.233284 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 510/1016] 10.222.0.18 () {56 vars in 1112 bytes} [Wed Aug 24 17:09:05 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:09:05.238663 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993e0618-271d-471a-862e-a754ac901258 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:05.338609 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993e0618-271d-471a-862e-a754ac901258 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:05.350314 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:09:05.374762 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03391db1-4404-4527-8bad-cfc310cd8deb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:05.375390 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03391db1-4404-4527-8bad-cfc310cd8deb None admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:05.375907 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03391db1-4404-4527-8bad-cfc310cd8deb None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:05.375907 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03391db1-4404-4527-8bad-cfc310cd8deb None admin] PATH_INFO: `/v3/endpoints` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:05.376878 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03391db1-4404-4527-8bad-cfc310cd8deb None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:05.378240 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03391db1-4404-4527-8bad-cfc310cd8deb None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:05.458498 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 507/1017] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 24 17:09:05 2022] POST /identity/v3/endpoints => generated 334 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:09:05.609906 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec4bd93-4f2d-496c-8d91-a7e144a251af None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:05.610463 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec4bd93-4f2d-496c-8d91-a7e144a251af None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:05.610463 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec4bd93-4f2d-496c-8d91-a7e144a251af None None] PATH_INFO: `/` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:05.612094 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 511/1018] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:09:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:09:05.617531 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a641695-e6cc-4348-a053-7e05b4a49b0b None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:05.617813 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a641695-e6cc-4348-a053-7e05b4a49b0b None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:05.618087 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a641695-e6cc-4348-a053-7e05b4a49b0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:05.638509 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-7a641695-e6cc-4348-a053-7e05b4a49b0b None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:09:05.639754 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-7a641695-e6cc-4348-a053-7e05b4a49b0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:09:05.717996 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 508/1019] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:09:05 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:09:05.987848 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec4bd93-4f2d-496c-8d91-a7e144a251af None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:06.050675 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:09:06.100883 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec4bd93-4f2d-496c-8d91-a7e144a251af None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:06.135154 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f6234b-1d9e-43c8-b5dc-63e600312cdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:06.135761 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f6234b-1d9e-43c8-b5dc-63e600312cdc None admin] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:06.136196 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f6234b-1d9e-43c8-b5dc-63e600312cdc None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:06.136469 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f6234b-1d9e-43c8-b5dc-63e600312cdc None admin] PATH_INFO: `/v3/services` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:06.137118 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f6234b-1d9e-43c8-b5dc-63e600312cdc None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:06.138514 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f6234b-1d9e-43c8-b5dc-63e600312cdc None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:06.234095 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 512/1020] 10.222.0.18 () {60 vars in 1141 bytes} [Wed Aug 24 17:09:05 2022] POST /identity/v3/services => generated 242 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:09:06.327028 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6637abef-5fc2-420c-988b-c98c94142254 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:06.327365 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6637abef-5fc2-420c-988b-c98c94142254 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:06.327619 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6637abef-5fc2-420c-988b-c98c94142254 None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:06.329119 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 509/1021] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:09:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:09:06.334573 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d64bb06-8611-4855-8a7b-ac501652b842 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:06.334858 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d64bb06-8611-4855-8a7b-ac501652b842 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:06.335154 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d64bb06-8611-4855-8a7b-ac501652b842 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:06.354221 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-5d64bb06-8611-4855-8a7b-ac501652b842 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:09:06.355859 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-5d64bb06-8611-4855-8a7b-ac501652b842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:09:06.439351 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 513/1022] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:09:06 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:09:06.706720 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6637abef-5fc2-420c-988b-c98c94142254 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:06.798756 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:09:06.809474 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6637abef-5fc2-420c-988b-c98c94142254 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:06.842706 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae6b42d-2067-4ef3-bd9c-8c6037485f39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:06.843388 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae6b42d-2067-4ef3-bd9c-8c6037485f39 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:06.843670 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae6b42d-2067-4ef3-bd9c-8c6037485f39 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:06.843946 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae6b42d-2067-4ef3-bd9c-8c6037485f39 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:06.845082 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae6b42d-2067-4ef3-bd9c-8c6037485f39 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:06.846519 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae6b42d-2067-4ef3-bd9c-8c6037485f39 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:06.848458 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 510/1023] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 24 17:09:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:09:06.853971 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d64bb06-8611-4855-8a7b-ac501652b842 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:06.901583 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:09:06.955691 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d64bb06-8611-4855-8a7b-ac501652b842 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:06.990917 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4558c3a2-3224-4fde-b6c7-ffef68a19052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:06.991442 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4558c3a2-3224-4fde-b6c7-ffef68a19052 None admin] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:06.991714 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4558c3a2-3224-4fde-b6c7-ffef68a19052 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:06.991989 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4558c3a2-3224-4fde-b6c7-ffef68a19052 None admin] PATH_INFO: `/v3/users` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:06.992639 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4558c3a2-3224-4fde-b6c7-ffef68a19052 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:06.994055 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4558c3a2-3224-4fde-b6c7-ffef68a19052 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:07.084490 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 514/1024] 10.222.0.18 () {60 vars in 1132 bytes} [Wed Aug 24 17:09:06 2022] POST /identity/v3/users => generated 285 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:09:07.466047 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490963fb-5248-47d1-8f82-18ae3463ec86 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:07.466424 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490963fb-5248-47d1-8f82-18ae3463ec86 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:07.466605 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490963fb-5248-47d1-8f82-18ae3463ec86 None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:07.468310 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 511/1025] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:09:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:09:07.473503 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1524ce68-dcdf-465f-891a-afedc6022e7a None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:07.473799 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1524ce68-dcdf-465f-891a-afedc6022e7a None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:07.474115 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1524ce68-dcdf-465f-891a-afedc6022e7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:07.494128 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-1524ce68-dcdf-465f-891a-afedc6022e7a None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:09:07.495878 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-1524ce68-dcdf-465f-891a-afedc6022e7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:09:07.564781 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 515/1026] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:09:07 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:09:07.835128 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490963fb-5248-47d1-8f82-18ae3463ec86 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:07.936310 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490963fb-5248-47d1-8f82-18ae3463ec86 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:07.969216 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e42c693-cfa9-44e3-b3cf-ff65f96d2df8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:07.969843 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e42c693-cfa9-44e3-b3cf-ff65f96d2df8 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:07.970117 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e42c693-cfa9-44e3-b3cf-ff65f96d2df8 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:07.970383 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e42c693-cfa9-44e3-b3cf-ff65f96d2df8 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:07.980896 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e42c693-cfa9-44e3-b3cf-ff65f96d2df8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:09:07.985463 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-5e42c693-cfa9-44e3-b3cf-ff65f96d2df8 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 17:09:07.985463 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:09:07.985463 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:09:07.985463 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:09:07.985463 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:09:07.985463 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:09:07.985463 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:09:07.985463 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:09:07.985463 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:09:07.985463 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:09:07.985463 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:09:07.985463 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:09:07.985463 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:09:07.985463 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:09:07.985463 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:09:07.985463 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:09:07.986377 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:09:07.986377 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:09:07.986377 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:09:07.986377 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:09:07.986377 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:09:07.986377 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:09:07.986377 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:09:07.986377 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:09:07.986377 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:09:07.986377 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:09:07.986377 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:09:07.986377 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 24 17:09:07.986377 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:09:07.986377 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:09:07.986377 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:09:07.986377 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:09:07.987904 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:09:07.987904 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:09:07.987904 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:09:07.987904 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:09:07.987904 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:09:07.987904 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:09:07.987904 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:09:07.987904 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:09:07.987904 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:09:07.987904 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:09:07.987904 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 17:09:07.987904 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:09:07.988932 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 512/1027] 10.222.0.18 () {56 vars in 1120 bytes} [Wed Aug 24 17:09:07 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:09:07.993698 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1524ce68-dcdf-465f-891a-afedc6022e7a None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:08.022684 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:09:08.098294 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1524ce68-dcdf-465f-891a-afedc6022e7a None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:08.134157 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56faeda-4cc4-451e-ae43-774c35c348da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:08.134769 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56faeda-4cc4-451e-ae43-774c35c348da None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:08.135073 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56faeda-4cc4-451e-ae43-774c35c348da None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:08.135344 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56faeda-4cc4-451e-ae43-774c35c348da None admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:08.136009 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56faeda-4cc4-451e-ae43-774c35c348da None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:09:08.136305 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56faeda-4cc4-451e-ae43-774c35c348da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:08.137658 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56faeda-4cc4-451e-ae43-774c35c348da None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:08.146034 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 516/1028] 10.222.0.18 () {56 vars in 1115 bytes} [Wed Aug 24 17:09:07 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:09:08.151450 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e42c693-cfa9-44e3-b3cf-ff65f96d2df8 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:08.175273 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:09:08.188871 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077cb021-a2e3-43f4-8be9-f2ea176d5e5c None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:08.189178 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077cb021-a2e3-43f4-8be9-f2ea176d5e5c None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:08.189524 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077cb021-a2e3-43f4-8be9-f2ea176d5e5c None None] PATH_INFO: `/` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:08.191078 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 517/1029] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:09:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 17:09:08.195894 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c7a223-0136-4871-a09e-6742bfc0ea3c None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:08.196182 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c7a223-0136-4871-a09e-6742bfc0ea3c None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:08.196459 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c7a223-0136-4871-a09e-6742bfc0ea3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:08.216249 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-28c7a223-0136-4871-a09e-6742bfc0ea3c None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:09:08.217498 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-28c7a223-0136-4871-a09e-6742bfc0ea3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:09:08.261498 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e42c693-cfa9-44e3-b3cf-ff65f96d2df8 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:08.288724 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 518/1030] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:09:08 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 17:09:08.300476 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6712a397-a60c-4174-a395-b99604a081e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:08.301104 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6712a397-a60c-4174-a395-b99604a081e7 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:08.301379 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6712a397-a60c-4174-a395-b99604a081e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:08.301647 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6712a397-a60c-4174-a395-b99604a081e7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:08.312449 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6712a397-a60c-4174-a395-b99604a081e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:08.313870 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6712a397-a60c-4174-a395-b99604a081e7 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:08.328646 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-6712a397-a60c-4174-a395-b99604a081e7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:09:08.328646 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:09:08.328646 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:09:08.328646 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:09:08.328646 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:09:08.328646 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:09:08.328646 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:09:08.328646 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:09:08.328646 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:09:08.328646 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:09:08.328646 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:09:08.328646 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:09:08.328646 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:09:08.328646 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:09:08.328646 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:09:08.328646 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:09:08.329599 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:09:08.329599 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:09:08.329599 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:09:08.329599 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:09:08.329599 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:09:08.329599 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:09:08.329599 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:09:08.329599 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 24 17:09:08.329599 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:09:08.329599 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:09:08.329599 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:09:08.329599 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:09:08.329599 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:09:08.329599 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:09:08.329599 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:09:08.329599 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:09:08.329599 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:09:08.330510 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:09:08.330510 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:09:08.330510 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:09:08.334605 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 513/1031] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 24 17:09:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:09:08.335866 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c7a223-0136-4871-a09e-6742bfc0ea3c None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:08.360534 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:09:08.436229 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c7a223-0136-4871-a09e-6742bfc0ea3c None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:08.470178 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc81a40-e525-46f2-9a1d-635d3cc162a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:08.470796 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc81a40-e525-46f2-9a1d-635d3cc162a2 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:08.471098 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc81a40-e525-46f2-9a1d-635d3cc162a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:08.471377 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc81a40-e525-46f2-9a1d-635d3cc162a2 None admin] PATH_INFO: `/v3/domains` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:08.472022 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc81a40-e525-46f2-9a1d-635d3cc162a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:09:08.472326 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc81a40-e525-46f2-9a1d-635d3cc162a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:08.473687 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc81a40-e525-46f2-9a1d-635d3cc162a2 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:08.489671 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 519/1032] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 24 17:09:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:09:08.490941 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6712a397-a60c-4174-a395-b99604a081e7 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:08.559469 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc81a40-e525-46f2-9a1d-635d3cc162a2 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:08.571617 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:09:08.604350 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6712a397-a60c-4174-a395-b99604a081e7 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:08.640822 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943e79e2-6369-4f4b-b58b-2ce4c217d088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:08.641377 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943e79e2-6369-4f4b-b58b-2ce4c217d088 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:08.641659 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943e79e2-6369-4f4b-b58b-2ce4c217d088 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:08.641932 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943e79e2-6369-4f4b-b58b-2ce4c217d088 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:08.649073 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:09:08.650055 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943e79e2-6369-4f4b-b58b-2ce4c217d088 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:08.651458 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943e79e2-6369-4f4b-b58b-2ce4c217d088 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:08.662481 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-943e79e2-6369-4f4b-b58b-2ce4c217d088 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 17:09:08.662481 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:09:08.662481 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:09:08.662481 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:09:08.662481 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:09:08.662481 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:09:08.662481 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:09:08.662481 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:09:08.662481 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:09:08.662481 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:09:08.662481 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:09:08.662481 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:09:08.662481 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:09:08.662481 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:09:08.662481 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:09:08.662481 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:09:08.663582 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:09:08.663582 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:09:08.663582 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:09:08.663582 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:09:08.663582 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:09:08.663582 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:09:08.663582 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:09:08.663582 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:09:08.663582 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:09:08.663582 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:09:08.663582 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:09:08.663582 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 24 17:09:08.663582 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:09:08.663582 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:09:08.663582 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:09:08.663582 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:09:08.663582 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:09:08.663582 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:09:08.669835 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:09:08.669835 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:09:08.669835 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:09:08.669835 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:09:08.669835 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:09:08.669835 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:09:08.669835 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:09:08.669835 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:09:08.669835 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 17:09:08.669835 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:09:08.669835 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 514/1033] 10.222.0.18 () {56 vars in 1093 bytes} [Wed Aug 24 17:09:08 2022] GET /identity/v3/users/nova => generated 82 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:09:08.669835 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943e79e2-6369-4f4b-b58b-2ce4c217d088 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:08.680337 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc81a40-e525-46f2-9a1d-635d3cc162a2 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:08.715381 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968019cd-1dcd-4f18-9973-9418fdfe519b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:08.716016 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968019cd-1dcd-4f18-9973-9418fdfe519b None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:08.716293 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968019cd-1dcd-4f18-9973-9418fdfe519b None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:08.716559 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968019cd-1dcd-4f18-9973-9418fdfe519b None admin] PATH_INFO: `/v3/services/image` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:08.717131 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968019cd-1dcd-4f18-9973-9418fdfe519b None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:08.718485 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968019cd-1dcd-4f18-9973-9418fdfe519b None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:08.727114 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-968019cd-1dcd-4f18-9973-9418fdfe519b None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 17:09:08.727114 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:09:08.727114 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 17:09:08.727114 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 17:09:08.727114 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 17:09:08.727114 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 17:09:08.727114 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 17:09:08.727114 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:09:08.727114 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 17:09:08.727114 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:09:08.727114 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 17:09:08.727114 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:09:08.727114 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:09:08.727114 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:09:08.727114 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:09:08.727114 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:09:08.727114 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:09:08.728331 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:09:08.728331 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:09:08.728331 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:09:08.728331 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:09:08.728331 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:09:08.728331 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:09:08.728331 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 17:09:08.728331 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 17:09:08.728331 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 17:09:08.728331 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 17:09:08.728331 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:09:08.728331 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:09:08.728331 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:09:08.728331 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:09:08.728331 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:09:08.728331 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:09:08.729403 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:09:08.729403 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application with Lock( Aug 24 17:09:08.729403 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:09:08.729403 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:09:08.729403 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:09:08.729403 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:09:08.729403 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:09:08.729403 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:09:08.729403 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:09:08.729403 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:09:08.729403 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 17:09:08.729403 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:09:08.729403 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 17:09:08.729403 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:09:08.729403 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 520/1034] 10.222.0.18 () {56 vars in 1105 bytes} [Wed Aug 24 17:09:08 2022] GET /identity/v3/services/image => generated 86 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:09:08.737673 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968019cd-1dcd-4f18-9973-9418fdfe519b None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:08.784167 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943e79e2-6369-4f4b-b58b-2ce4c217d088 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:08.814283 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:09:08.819770 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54541d83-5ab6-4e95-b767-5b100669fdb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:08.820389 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54541d83-5ab6-4e95-b767-5b100669fdb9 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:08.822960 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54541d83-5ab6-4e95-b767-5b100669fdb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:08.822960 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54541d83-5ab6-4e95-b767-5b100669fdb9 None admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:08.822960 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54541d83-5ab6-4e95-b767-5b100669fdb9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:09:08.822960 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54541d83-5ab6-4e95-b767-5b100669fdb9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:08.823452 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54541d83-5ab6-4e95-b767-5b100669fdb9 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:08.839942 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968019cd-1dcd-4f18-9973-9418fdfe519b None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:08.845992 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 515/1035] 10.222.0.18 () {56 vars in 1133 bytes} [Wed Aug 24 17:09:08 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:09:08.847021 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54541d83-5ab6-4e95-b767-5b100669fdb9 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:08.881688 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e62cdb-2f44-4b36-882a-3756a61a2aa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:08.882635 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e62cdb-2f44-4b36-882a-3756a61a2aa3 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:08.883127 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e62cdb-2f44-4b36-882a-3756a61a2aa3 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:08.883586 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e62cdb-2f44-4b36-882a-3756a61a2aa3 None admin] PATH_INFO: `/v3/services` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:08.884414 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e62cdb-2f44-4b36-882a-3756a61a2aa3 None admin] RBAC: Adding query filter params (name=image) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:09:08.884906 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e62cdb-2f44-4b36-882a-3756a61a2aa3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:08.886609 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e62cdb-2f44-4b36-882a-3756a61a2aa3 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:08.892426 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 521/1036] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 24 17:09:08 2022] GET /identity/v3/services?name=image => generated 122 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:09:08.897765 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e62cdb-2f44-4b36-882a-3756a61a2aa3 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:08.911131 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:09:08.949868 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54541d83-5ab6-4e95-b767-5b100669fdb9 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:08.984441 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04960a8-5129-45a7-bafe-9c5da9b56c89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:08.985055 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04960a8-5129-45a7-bafe-9c5da9b56c89 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:08.985336 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04960a8-5129-45a7-bafe-9c5da9b56c89 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:08.985609 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04960a8-5129-45a7-bafe-9c5da9b56c89 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:09.005658 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e62cdb-2f44-4b36-882a-3756a61a2aa3 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:09.044892 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfd1bdd-4a21-4a14-8049-a8f06857a8e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:09.045616 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfd1bdd-4a21-4a14-8049-a8f06857a8e6 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:09.045897 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfd1bdd-4a21-4a14-8049-a8f06857a8e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:09.046291 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfd1bdd-4a21-4a14-8049-a8f06857a8e6 None admin] PATH_INFO: `/v3/services` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:09.046941 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfd1bdd-4a21-4a14-8049-a8f06857a8e6 None admin] RBAC: Adding query filter params (type=image) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:09:09.047250 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfd1bdd-4a21-4a14-8049-a8f06857a8e6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:09.048901 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:09:09.049010 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfd1bdd-4a21-4a14-8049-a8f06857a8e6 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:09.059967 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 522/1037] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 24 17:09:08 2022] GET /identity/v3/services?type=image => generated 350 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:09:09.059967 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2ba42f-4bf4-43b8-914c-425329ec823c None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:09.059967 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2ba42f-4bf4-43b8-914c-425329ec823c None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:09.059967 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2ba42f-4bf4-43b8-914c-425329ec823c None None] PATH_INFO: `/` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:09.059967 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 523/1038] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:09:09 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:09:09.063413 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04960a8-5129-45a7-bafe-9c5da9b56c89 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:09.063512 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2ba42f-4bf4-43b8-914c-425329ec823c None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:09.063594 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04960a8-5129-45a7-bafe-9c5da9b56c89 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:09.140123 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:09:09.163774 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 516/1039] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 24 17:09:08 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 17:09:09.164931 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184d2fc5-81cf-4676-bb76-50611faefea6 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:09.165220 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184d2fc5-81cf-4676-bb76-50611faefea6 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:09.165492 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184d2fc5-81cf-4676-bb76-50611faefea6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:09.170567 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2ba42f-4bf4-43b8-914c-425329ec823c None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:09.183579 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-184d2fc5-81cf-4676-bb76-50611faefea6 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:09:09.184777 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-184d2fc5-81cf-4676-bb76-50611faefea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:09:09.205062 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8832d452-50a1-4d50-8914-8dec3c30e30e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:09.205666 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8832d452-50a1-4d50-8914-8dec3c30e30e None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:09.205948 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8832d452-50a1-4d50-8914-8dec3c30e30e None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:09.206217 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8832d452-50a1-4d50-8914-8dec3c30e30e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:09.206924 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8832d452-50a1-4d50-8914-8dec3c30e30e None admin] RBAC: Adding query filter params (interface=public, service_id=4b7fe4ab82334485b066a70962ab8cc6, region_id=RegionOne) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:09:09.207239 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8832d452-50a1-4d50-8914-8dec3c30e30e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:09.208583 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8832d452-50a1-4d50-8914-8dec3c30e30e None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:09.213773 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 524/1040] 10.222.0.18 () {56 vars in 1251 bytes} [Wed Aug 24 17:09:09 2022] GET /identity/v3/endpoints?service_id=4b7fe4ab82334485b066a70962ab8cc6&interface=public®ion_id=RegionOne => generated 194 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:09:09.214316 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8832d452-50a1-4d50-8914-8dec3c30e30e None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:09.250658 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 517/1041] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:09:09 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:09:09.252135 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184d2fc5-81cf-4676-bb76-50611faefea6 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:09.274523 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:09:09.317559 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8832d452-50a1-4d50-8914-8dec3c30e30e None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:09.358528 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184d2fc5-81cf-4676-bb76-50611faefea6 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:09.363615 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8958e80b-29d8-4d66-bcb2-288a1d178271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:09.366487 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8958e80b-29d8-4d66-bcb2-288a1d178271 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:09.366487 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8958e80b-29d8-4d66-bcb2-288a1d178271 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:09.366487 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8958e80b-29d8-4d66-bcb2-288a1d178271 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:09.375115 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:09:09.380553 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8958e80b-29d8-4d66-bcb2-288a1d178271 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:09.382270 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8958e80b-29d8-4d66-bcb2-288a1d178271 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:09.398365 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bafe52-e218-4bdf-9188-5b09249676e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:09.399023 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bafe52-e218-4bdf-9188-5b09249676e0 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:09.399308 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bafe52-e218-4bdf-9188-5b09249676e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:09.399586 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bafe52-e218-4bdf-9188-5b09249676e0 None admin] PATH_INFO: `/v3/services` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:09.400200 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bafe52-e218-4bdf-9188-5b09249676e0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:09.400591 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-8958e80b-29d8-4d66-bcb2-288a1d178271 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:09:09.400591 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:09:09.400591 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:09:09.400591 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:09:09.400591 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:09:09.400591 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:09:09.400591 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:09:09.400591 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:09:09.400591 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:09:09.400591 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:09:09.400591 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:09:09.400591 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:09:09.400591 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:09:09.400591 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:09:09.400591 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:09:09.400591 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:09:09.401532 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:09:09.401532 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:09:09.401532 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:09:09.401532 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:09:09.401532 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:09:09.401532 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:09:09.401532 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:09:09.401532 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application with Lock( Aug 24 17:09:09.401532 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:09:09.401532 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:09:09.401532 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:09:09.401532 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:09:09.401532 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:09:09.401532 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:09:09.401532 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:09:09.401532 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:09:09.401532 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:09:09.402436 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:09:09.402436 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:09:09.402436 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:09:09.402620 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bafe52-e218-4bdf-9188-5b09249676e0 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:09.409009 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 525/1042] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 24 17:09:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:09:09.410031 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8958e80b-29d8-4d66-bcb2-288a1d178271 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:09.413172 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 518/1043] 10.222.0.18 () {56 vars in 1087 bytes} [Wed Aug 24 17:09:09 2022] GET /identity/v3/services => generated 1957 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:09:09.521907 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8958e80b-29d8-4d66-bcb2-288a1d178271 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:09.528901 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bafe52-e218-4bdf-9188-5b09249676e0 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:09.566169 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e974cdbb-16f8-4b4a-ae12-5ee32fa67e62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:09.566791 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e974cdbb-16f8-4b4a-ae12-5ee32fa67e62 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:09.567331 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e974cdbb-16f8-4b4a-ae12-5ee32fa67e62 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:09.568310 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e974cdbb-16f8-4b4a-ae12-5ee32fa67e62 None admin] PATH_INFO: `/v3/domains` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:09.568800 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e974cdbb-16f8-4b4a-ae12-5ee32fa67e62 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:09:09.569284 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e974cdbb-16f8-4b4a-ae12-5ee32fa67e62 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:09.571675 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e974cdbb-16f8-4b4a-ae12-5ee32fa67e62 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:09.586614 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 526/1044] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 24 17:09:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:09:09.594994 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e974cdbb-16f8-4b4a-ae12-5ee32fa67e62 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:09.627871 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:09:09.645096 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bafe52-e218-4bdf-9188-5b09249676e0 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:09.649349 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:09:09.688411 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4f45c8-d19f-4ea0-a684-0f00a4bc8940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:09.688866 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4f45c8-d19f-4ea0-a684-0f00a4bc8940 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:09.689691 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4f45c8-d19f-4ea0-a684-0f00a4bc8940 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:09.690979 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4f45c8-d19f-4ea0-a684-0f00a4bc8940 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:09.703312 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4f45c8-d19f-4ea0-a684-0f00a4bc8940 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:09:09.708036 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-5d4f45c8-d19f-4ea0-a684-0f00a4bc8940 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:09:09.708036 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:09:09.708036 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:09:09.708036 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:09:09.708036 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:09:09.708036 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:09:09.708036 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:09:09.708036 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:09:09.708036 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:09:09.708036 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:09:09.708036 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:09:09.708036 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:09:09.708036 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:09:09.708036 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:09:09.708036 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:09:09.708036 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:09:09.708979 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:09:09.708979 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:09:09.708979 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:09:09.708979 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:09:09.708979 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:09:09.708979 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:09:09.708979 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:09:09.708979 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:09:09.708979 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:09:09.708979 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:09:09.708979 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:09:09.708979 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 24 17:09:09.708979 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:09:09.708979 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:09:09.708979 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:09:09.708979 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:09:09.709842 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:09:09.709842 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:09:09.709842 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:09:09.709842 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:09:09.709842 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:09:09.709842 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:09:09.709842 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:09:09.709842 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:09:09.709842 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:09:09.709842 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:09:09.709842 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:09:09.709842 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:09:09.710530 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e974cdbb-16f8-4b4a-ae12-5ee32fa67e62 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:09.710608 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 519/1045] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Aug 24 17:09:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:09:09.715457 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4f45c8-d19f-4ea0-a684-0f00a4bc8940 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:09.750829 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbdae45-1257-4dd0-a187-82ddc2f4a291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:09.752015 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbdae45-1257-4dd0-a187-82ddc2f4a291 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:09.752301 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbdae45-1257-4dd0-a187-82ddc2f4a291 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:09.752631 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbdae45-1257-4dd0-a187-82ddc2f4a291 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:09.759628 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbdae45-1257-4dd0-a187-82ddc2f4a291 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:09.761111 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbdae45-1257-4dd0-a187-82ddc2f4a291 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:09.772347 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-dbbdae45-1257-4dd0-a187-82ddc2f4a291 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:09:09.772347 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:09:09.772347 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:09:09.772347 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:09:09.772347 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:09:09.772347 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:09:09.772347 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:09:09.772347 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:09:09.772347 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:09:09.772347 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:09:09.772347 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:09:09.772347 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:09:09.772347 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:09:09.772347 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:09:09.772347 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:09:09.773261 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:09:09.773261 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:09:09.773261 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:09:09.773261 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:09:09.773261 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:09:09.773261 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:09:09.773261 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:09:09.773261 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:09:09.773261 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application with Lock( Aug 24 17:09:09.773261 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:09:09.773261 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:09:09.773261 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:09:09.773261 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:09:09.773261 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:09:09.773261 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:09:09.773261 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:09:09.773261 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:09:09.774173 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:09:09.774173 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:09:09.774173 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:09:09.774173 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:09:09.774173 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:09:09.774173 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:09:09.774173 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:09:09.774173 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:09:09.774173 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 527/1046] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 24 17:09:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:09:09.780026 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbdae45-1257-4dd0-a187-82ddc2f4a291 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:09.830397 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4f45c8-d19f-4ea0-a684-0f00a4bc8940 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:09.867091 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be6e2e1-b237-4602-85c5-4569318e6bbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:09.867742 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be6e2e1-b237-4602-85c5-4569318e6bbc None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:09.867971 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be6e2e1-b237-4602-85c5-4569318e6bbc None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:09.868242 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be6e2e1-b237-4602-85c5-4569318e6bbc None admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:09.868909 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be6e2e1-b237-4602-85c5-4569318e6bbc None admin] RBAC: Adding query filter params (name=admin) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:09:09.869216 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be6e2e1-b237-4602-85c5-4569318e6bbc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:09.871269 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be6e2e1-b237-4602-85c5-4569318e6bbc None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:09.882126 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 520/1047] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 24 17:09:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:09:09.882679 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbdae45-1257-4dd0-a187-82ddc2f4a291 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:09.891964 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be6e2e1-b237-4602-85c5-4569318e6bbc None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:09.896485 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:09:09.901219 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:09:09.920632 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a110dae5-a9d7-4ca5-aac8-2d9ed99271b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:09.921238 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a110dae5-a9d7-4ca5-aac8-2d9ed99271b6 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:09.921520 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a110dae5-a9d7-4ca5-aac8-2d9ed99271b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:09.921785 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a110dae5-a9d7-4ca5-aac8-2d9ed99271b6 None admin] PATH_INFO: `/v3/projects` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:09.922536 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a110dae5-a9d7-4ca5-aac8-2d9ed99271b6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:09:09.922841 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a110dae5-a9d7-4ca5-aac8-2d9ed99271b6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:09.924284 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a110dae5-a9d7-4ca5-aac8-2d9ed99271b6 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:09.941944 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 528/1048] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Aug 24 17:09:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:09:09.946354 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a110dae5-a9d7-4ca5-aac8-2d9ed99271b6 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:10.002645 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be6e2e1-b237-4602-85c5-4569318e6bbc None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:10.045843 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7998233-3ce3-4275-8013-aa01e6e7e61d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:10.046400 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7998233-3ce3-4275-8013-aa01e6e7e61d None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:10.046674 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7998233-3ce3-4275-8013-aa01e6e7e61d None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:10.046962 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7998233-3ce3-4275-8013-aa01e6e7e61d None admin] PATH_INFO: `/v3/users/e97038018a5e422a83f2235bd1240bf7` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:10.056609 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a110dae5-a9d7-4ca5-aac8-2d9ed99271b6 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:10.062577 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7998233-3ce3-4275-8013-aa01e6e7e61d None admin] RBAC: Authorizing `identity:get_user(user_id=e97038018a5e422a83f2235bd1240bf7)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:10.064057 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7998233-3ce3-4275-8013-aa01e6e7e61d None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:10.070748 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 521/1049] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 24 17:09:09 2022] GET /identity/v3/users/e97038018a5e422a83f2235bd1240bf7 => generated 285 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:09:10.071865 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7998233-3ce3-4275-8013-aa01e6e7e61d None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:10.095559 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5b1130-4120-4a71-8d07-caba9971000e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:10.096170 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5b1130-4120-4a71-8d07-caba9971000e None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:10.096491 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5b1130-4120-4a71-8d07-caba9971000e None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:10.096763 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5b1130-4120-4a71-8d07-caba9971000e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:10.097483 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5b1130-4120-4a71-8d07-caba9971000e None admin] RBAC: Adding query filter params (role.id=967c5aaa9ba746a7acaa59f3e9691b31, scope.project.id=46ab158bc3644c888557781b0630b0b8, user.id=557aca9255cb4bbb8af52e796c5c5177) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:09:10.097786 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5b1130-4120-4a71-8d07-caba9971000e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:10.099221 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5b1130-4120-4a71-8d07-caba9971000e None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:10.118019 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 529/1050] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 24 17:09:09 2022] GET /identity/v3/role_assignments?user.id=557aca9255cb4bbb8af52e796c5c5177&scope.project.id=46ab158bc3644c888557781b0630b0b8&role.id=967c5aaa9ba746a7acaa59f3e9691b31 => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:09:10.141474 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:09:10.183605 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7998233-3ce3-4275-8013-aa01e6e7e61d None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:10.221526 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd4fcd6-1078-4bd9-84d4-70357e43a2a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:10.222329 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd4fcd6-1078-4bd9-84d4-70357e43a2a4 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:10.222624 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd4fcd6-1078-4bd9-84d4-70357e43a2a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:10.222963 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd4fcd6-1078-4bd9-84d4-70357e43a2a4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:10.296959 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd4fcd6-1078-4bd9-84d4-70357e43a2a4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:10.298330 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd4fcd6-1078-4bd9-84d4-70357e43a2a4 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:10.360847 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:09:10.405429 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 522/1051] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 24 17:09:10 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 17:09:10.406689 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5b1130-4120-4a71-8d07-caba9971000e None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:10.443264 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:09:10.509525 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5b1130-4120-4a71-8d07-caba9971000e None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:10.543903 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1515ef61-d964-4f02-9881-ff61a520350f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:10.544558 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1515ef61-d964-4f02-9881-ff61a520350f None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:10.544836 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1515ef61-d964-4f02-9881-ff61a520350f None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:10.545108 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1515ef61-d964-4f02-9881-ff61a520350f None admin] PATH_INFO: `/v3/projects/408deae9dd0b48e79cc6b850a2f006b7` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:10.546303 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1515ef61-d964-4f02-9881-ff61a520350f None admin] RBAC: Authorizing `identity:get_project(project_id=408deae9dd0b48e79cc6b850a2f006b7)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:10.547846 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1515ef61-d964-4f02-9881-ff61a520350f None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:10.549650 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 530/1052] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 24 17:09:10 2022] GET /identity/v3/projects/408deae9dd0b48e79cc6b850a2f006b7 => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:09:10.558832 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd4fcd6-1078-4bd9-84d4-70357e43a2a4 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:10.665255 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd4fcd6-1078-4bd9-84d4-70357e43a2a4 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:10.700271 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9e0ceb-6a3e-402e-9365-c2587460fb7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:10.700898 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9e0ceb-6a3e-402e-9365-c2587460fb7c None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:10.701182 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9e0ceb-6a3e-402e-9365-c2587460fb7c None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:10.701457 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9e0ceb-6a3e-402e-9365-c2587460fb7c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:10.702180 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9e0ceb-6a3e-402e-9365-c2587460fb7c None admin] RBAC: Adding query filter params (role.id=c5346bc54abf445f9900edf94264c850, scope.project.id=408deae9dd0b48e79cc6b850a2f006b7, user.id=e97038018a5e422a83f2235bd1240bf7) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:09:10.702491 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9e0ceb-6a3e-402e-9365-c2587460fb7c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:10.703949 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9e0ceb-6a3e-402e-9365-c2587460fb7c None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:10.709434 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 523/1053] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 24 17:09:10 2022] GET /identity/v3/role_assignments?user.id=e97038018a5e422a83f2235bd1240bf7&scope.project.id=408deae9dd0b48e79cc6b850a2f006b7&role.id=c5346bc54abf445f9900edf94264c850 => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:09:11.384982 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92ac239-3c69-457b-ad81-b83f50785f5e None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:11.385549 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92ac239-3c69-457b-ad81-b83f50785f5e None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:11.385549 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92ac239-3c69-457b-ad81-b83f50785f5e None None] PATH_INFO: `/` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:11.387013 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 531/1054] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:09:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:09:11.392059 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0300f0-f0a3-47b8-a4db-df04afc3b727 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:11.393019 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0300f0-f0a3-47b8-a4db-df04afc3b727 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:11.393019 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0300f0-f0a3-47b8-a4db-df04afc3b727 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:11.410619 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-fc0300f0-f0a3-47b8-a4db-df04afc3b727 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:09:11.411801 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-fc0300f0-f0a3-47b8-a4db-df04afc3b727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:09:11.434963 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:09:11.481191 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 524/1055] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:09:11 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:09:11.750412 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92ac239-3c69-457b-ad81-b83f50785f5e None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:11.857941 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92ac239-3c69-457b-ad81-b83f50785f5e None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:11.859426 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:09:11.894013 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a14a14-60cf-40bf-9115-534c23dc49ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:11.894650 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a14a14-60cf-40bf-9115-534c23dc49ee None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:11.894942 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a14a14-60cf-40bf-9115-534c23dc49ee None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:11.895213 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a14a14-60cf-40bf-9115-534c23dc49ee None admin] PATH_INFO: `/v3/services/image` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:11.895788 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a14a14-60cf-40bf-9115-534c23dc49ee None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:11.897137 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a14a14-60cf-40bf-9115-534c23dc49ee None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:11.905881 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-36a14a14-60cf-40bf-9115-534c23dc49ee None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 17:09:11.905881 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:09:11.905881 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 17:09:11.905881 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 17:09:11.905881 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 17:09:11.905881 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 17:09:11.905881 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 17:09:11.905881 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:09:11.905881 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 17:09:11.905881 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:09:11.905881 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 17:09:11.905881 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:09:11.905881 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:09:11.905881 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:09:11.905881 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:09:11.905881 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:09:11.905881 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:09:11.906842 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:09:11.906842 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:09:11.906842 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:09:11.906842 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:09:11.906842 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:09:11.906842 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:09:11.906842 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 17:09:11.906842 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 17:09:11.906842 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 17:09:11.906842 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 17:09:11.906842 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:09:11.906842 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:09:11.906842 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:09:11.906842 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:09:11.906842 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:09:11.906842 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:09:11.912251 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:09:11.912251 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application with Lock( Aug 24 17:09:11.912251 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:09:11.912251 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:09:11.912251 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:09:11.912251 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:09:11.912251 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:09:11.912251 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:09:11.912251 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:09:11.912251 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:09:11.912251 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 17:09:11.912251 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:09:11.912251 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 17:09:11.912251 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:09:11.912251 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 532/1056] 10.222.0.18 () {56 vars in 1105 bytes} [Wed Aug 24 17:09:11 2022] GET /identity/v3/services/image => generated 86 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:09:11.913060 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0300f0-f0a3-47b8-a4db-df04afc3b727 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:12.014668 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0300f0-f0a3-47b8-a4db-df04afc3b727 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:12.027998 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aefef8-18af-46b1-8400-a9a04cdb9fa8 None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:12.028383 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aefef8-18af-46b1-8400-a9a04cdb9fa8 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:12.028823 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aefef8-18af-46b1-8400-a9a04cdb9fa8 None None] PATH_INFO: `/` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:12.030257 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 533/1057] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:09:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 17:09:12.035404 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fae48f-2c5e-4607-9ec6-e4960953a4b7 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:12.037296 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fae48f-2c5e-4607-9ec6-e4960953a4b7 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:12.037582 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fae48f-2c5e-4607-9ec6-e4960953a4b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:12.051801 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b15929-2bec-4cbe-a13d-440333a38d2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:12.052482 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b15929-2bec-4cbe-a13d-440333a38d2a None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:12.052728 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b15929-2bec-4cbe-a13d-440333a38d2a None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:12.052955 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b15929-2bec-4cbe-a13d-440333a38d2a None admin] PATH_INFO: `/v3/services` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:12.053587 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b15929-2bec-4cbe-a13d-440333a38d2a None admin] RBAC: Adding query filter params (name=image) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:09:12.053888 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b15929-2bec-4cbe-a13d-440333a38d2a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:12.055288 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b15929-2bec-4cbe-a13d-440333a38d2a None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:12.056608 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-f4fae48f-2c5e-4607-9ec6-e4960953a4b7 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:09:12.058364 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-f4fae48f-2c5e-4607-9ec6-e4960953a4b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:09:12.060959 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 525/1058] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 24 17:09:11 2022] GET /identity/v3/services?name=image => generated 122 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:09:12.066932 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b15929-2bec-4cbe-a13d-440333a38d2a None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:12.091299 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:09:12.139499 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 534/1059] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:09:12 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 17:09:12.173749 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b15929-2bec-4cbe-a13d-440333a38d2a None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:12.208613 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df26e35-c9e5-4f2c-9b8f-3143f014cfce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:12.209218 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df26e35-c9e5-4f2c-9b8f-3143f014cfce None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:12.209496 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df26e35-c9e5-4f2c-9b8f-3143f014cfce None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:12.209767 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df26e35-c9e5-4f2c-9b8f-3143f014cfce None admin] PATH_INFO: `/v3/services` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:12.210392 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df26e35-c9e5-4f2c-9b8f-3143f014cfce None admin] RBAC: Adding query filter params (type=image) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:09:12.210694 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df26e35-c9e5-4f2c-9b8f-3143f014cfce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:12.212124 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df26e35-c9e5-4f2c-9b8f-3143f014cfce None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:12.221745 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 526/1060] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 24 17:09:12 2022] GET /identity/v3/services?type=image => generated 350 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:09:12.222754 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fae48f-2c5e-4607-9ec6-e4960953a4b7 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:12.317633 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:09:12.324259 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fae48f-2c5e-4607-9ec6-e4960953a4b7 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:12.357218 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cc9c3b-2dec-45c7-9140-78aa032e585b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:12.357824 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cc9c3b-2dec-45c7-9140-78aa032e585b None admin] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:12.358098 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cc9c3b-2dec-45c7-9140-78aa032e585b None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:12.358365 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cc9c3b-2dec-45c7-9140-78aa032e585b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:12.359098 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cc9c3b-2dec-45c7-9140-78aa032e585b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:12.360444 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cc9c3b-2dec-45c7-9140-78aa032e585b None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:12.418516 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df26e35-c9e5-4f2c-9b8f-3143f014cfce None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:12.440289 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 535/1061] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 24 17:09:12 2022] POST /identity/v3/endpoints => generated 327 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:09:12.514881 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:09:12.532015 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df26e35-c9e5-4f2c-9b8f-3143f014cfce None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:12.580338 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427b9b27-894a-4f68-9c5c-1f9d5aca3888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:12.580894 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427b9b27-894a-4f68-9c5c-1f9d5aca3888 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:12.581168 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427b9b27-894a-4f68-9c5c-1f9d5aca3888 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:12.581433 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427b9b27-894a-4f68-9c5c-1f9d5aca3888 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:12.592802 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427b9b27-894a-4f68-9c5c-1f9d5aca3888 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:09:12.598075 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-427b9b27-894a-4f68-9c5c-1f9d5aca3888 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 17:09:12.598075 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:09:12.598075 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:09:12.598075 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:09:12.598075 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:09:12.598075 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:09:12.598075 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:09:12.598075 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:09:12.598075 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:09:12.598075 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:09:12.598075 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:09:12.598075 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:09:12.598075 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:09:12.598075 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:09:12.598075 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:09:12.598075 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:09:12.599069 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:09:12.599069 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:09:12.599069 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:09:12.599069 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:09:12.599069 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:09:12.599069 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:09:12.599069 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:09:12.599069 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:09:12.599069 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:09:12.599069 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:09:12.599069 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:09:12.599069 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 24 17:09:12.599069 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:09:12.599069 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:09:12.599069 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:09:12.599069 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:09:12.600129 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:09:12.600129 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:09:12.600129 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:09:12.600129 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:09:12.600129 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:09:12.600129 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:09:12.600129 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:09:12.600129 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:09:12.600129 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:09:12.600129 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:09:12.600129 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 17:09:12.600129 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:09:12.602091 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 527/1062] 10.222.0.18 () {56 vars in 1120 bytes} [Wed Aug 24 17:09:12 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:09:12.607137 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e37dade-de60-409d-97f9-276ba33d2db3 None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:12.607672 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e37dade-de60-409d-97f9-276ba33d2db3 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:12.607958 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e37dade-de60-409d-97f9-276ba33d2db3 None None] PATH_INFO: `/` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:12.608422 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427b9b27-894a-4f68-9c5c-1f9d5aca3888 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:12.609280 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 536/1063] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:09:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:09:12.614337 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69d10ed-58e0-48a9-b80e-9b16217cbc1e None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:12.614691 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69d10ed-58e0-48a9-b80e-9b16217cbc1e None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:12.615085 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69d10ed-58e0-48a9-b80e-9b16217cbc1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:12.637788 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-d69d10ed-58e0-48a9-b80e-9b16217cbc1e None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:09:12.639012 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-d69d10ed-58e0-48a9-b80e-9b16217cbc1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:09:12.710754 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 537/1064] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:09:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 17:09:12.723036 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427b9b27-894a-4f68-9c5c-1f9d5aca3888 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:12.769273 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936985a2-8202-48c9-9208-52fab7ccabf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:12.770285 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936985a2-8202-48c9-9208-52fab7ccabf9 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:12.772024 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936985a2-8202-48c9-9208-52fab7ccabf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:12.772489 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936985a2-8202-48c9-9208-52fab7ccabf9 None admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:12.774422 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936985a2-8202-48c9-9208-52fab7ccabf9 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:09:12.774963 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936985a2-8202-48c9-9208-52fab7ccabf9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:12.777645 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936985a2-8202-48c9-9208-52fab7ccabf9 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:12.787932 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 528/1065] 10.222.0.18 () {56 vars in 1115 bytes} [Wed Aug 24 17:09:12 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:09:12.793313 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69d10ed-58e0-48a9-b80e-9b16217cbc1e None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:12.876353 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:09:12.894868 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69d10ed-58e0-48a9-b80e-9b16217cbc1e None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:12.931136 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0848daf-594b-4c98-be2c-7a5f190757c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:12.931771 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0848daf-594b-4c98-be2c-7a5f190757c8 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:12.932050 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0848daf-594b-4c98-be2c-7a5f190757c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:12.932321 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0848daf-594b-4c98-be2c-7a5f190757c8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:12.943942 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0848daf-594b-4c98-be2c-7a5f190757c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:12.945377 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0848daf-594b-4c98-be2c-7a5f190757c8 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:12.959774 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-c0848daf-594b-4c98-be2c-7a5f190757c8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:09:12.959774 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:09:12.959774 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:09:12.959774 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:09:12.959774 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:09:12.959774 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:09:12.959774 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:09:12.959774 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:09:12.959774 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:09:12.959774 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:09:12.959774 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:09:12.959774 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:09:12.959774 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:09:12.959774 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:09:12.959774 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:09:12.959774 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:09:12.960772 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:09:12.960772 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:09:12.960772 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:09:12.960772 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:09:12.960772 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:09:12.960772 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:09:12.960772 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:09:12.960772 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application with Lock( Aug 24 17:09:12.960772 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:09:12.960772 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:09:12.960772 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:09:12.960772 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:09:12.960772 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:09:12.960772 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:09:12.960772 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:09:12.960772 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:09:12.960772 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:09:12.961679 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:09:12.961679 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:09:12.961679 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:09:12.962570 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 538/1066] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 24 17:09:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:09:12.966632 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936985a2-8202-48c9-9208-52fab7ccabf9 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:12.971185 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:09:12.983981 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0848daf-594b-4c98-be2c-7a5f190757c8 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:13.084075 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936985a2-8202-48c9-9208-52fab7ccabf9 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:13.099067 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0848daf-594b-4c98-be2c-7a5f190757c8 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:13.121322 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e098cf-50e1-4522-8eea-27ec8523f55a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:13.121930 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e098cf-50e1-4522-8eea-27ec8523f55a None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:13.122211 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e098cf-50e1-4522-8eea-27ec8523f55a None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:13.122483 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e098cf-50e1-4522-8eea-27ec8523f55a None admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:13.123159 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e098cf-50e1-4522-8eea-27ec8523f55a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:09:13.123454 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e098cf-50e1-4522-8eea-27ec8523f55a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:13.124815 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e098cf-50e1-4522-8eea-27ec8523f55a None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:13.137760 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 529/1067] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 24 17:09:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:09:13.137899 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2faed63-d995-4697-8c07-453fb41a383c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:13.138426 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2faed63-d995-4697-8c07-453fb41a383c None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:13.138783 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2faed63-d995-4697-8c07-453fb41a383c None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:13.139092 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2faed63-d995-4697-8c07-453fb41a383c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:13.140991 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:09:13.143258 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e098cf-50e1-4522-8eea-27ec8523f55a None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:13.151529 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2faed63-d995-4697-8c07-453fb41a383c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:09:13.156465 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-e2faed63-d995-4697-8c07-453fb41a383c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:09:13.156465 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:09:13.156465 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:09:13.156465 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:09:13.156465 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:09:13.156465 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:09:13.156465 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:09:13.156465 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:09:13.156465 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:09:13.156465 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:09:13.156465 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:09:13.156465 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:09:13.156465 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:09:13.156465 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:09:13.156465 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:09:13.156465 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:09:13.157392 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:09:13.157392 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:09:13.157392 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:09:13.157392 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:09:13.157392 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:09:13.157392 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:09:13.157392 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:09:13.157392 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:09:13.157392 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:09:13.157392 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:09:13.157392 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:09:13.157392 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application with Lock( Aug 24 17:09:13.157392 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:09:13.157392 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:09:13.157392 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:09:13.157392 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:09:13.158230 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:09:13.158230 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:09:13.158230 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:09:13.158230 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:09:13.158230 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:09:13.158230 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:09:13.158230 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:09:13.158230 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:09:13.158230 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:09:13.158230 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:09:13.158230 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:09:13.158230 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:09:13.158230 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 539/1068] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Aug 24 17:09:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:09:13.163415 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2faed63-d995-4697-8c07-453fb41a383c None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:13.210666 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:09:13.249111 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e098cf-50e1-4522-8eea-27ec8523f55a None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:13.267962 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2faed63-d995-4697-8c07-453fb41a383c None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:13.285899 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e176cb9-3b0e-4011-86d0-aaf6553d4bff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:13.286523 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e176cb9-3b0e-4011-86d0-aaf6553d4bff None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:13.286801 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e176cb9-3b0e-4011-86d0-aaf6553d4bff None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:13.287099 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e176cb9-3b0e-4011-86d0-aaf6553d4bff None admin] PATH_INFO: `/v3/users/nova` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:13.294766 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e176cb9-3b0e-4011-86d0-aaf6553d4bff None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:13.296239 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e176cb9-3b0e-4011-86d0-aaf6553d4bff None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:13.308128 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-4e176cb9-3b0e-4011-86d0-aaf6553d4bff None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 17:09:13.308128 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:09:13.308128 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:09:13.308128 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:09:13.308128 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:09:13.308128 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:09:13.308128 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:09:13.308128 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:09:13.308128 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:09:13.308128 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:09:13.308128 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:09:13.308128 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:09:13.308128 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:09:13.308128 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:09:13.308128 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:09:13.308128 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:09:13.309528 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:09:13.309528 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:09:13.309528 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:09:13.309528 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:09:13.309528 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:09:13.309528 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:09:13.309528 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:09:13.309528 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:09:13.309528 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:09:13.309528 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:09:13.309528 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:09:13.309528 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 24 17:09:13.309528 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:09:13.309528 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:09:13.309528 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:09:13.309528 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:09:13.309528 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:09:13.309528 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:09:13.310996 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:09:13.310996 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:09:13.310996 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:09:13.310996 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:09:13.310996 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:09:13.310996 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:09:13.310996 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:09:13.310996 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:09:13.310996 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 17:09:13.310996 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:09:13.311548 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec6c4f1-b662-4561-a1e7-a92522a08819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:13.311641 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec6c4f1-b662-4561-a1e7-a92522a08819 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:13.311947 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec6c4f1-b662-4561-a1e7-a92522a08819 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:13.312130 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec6c4f1-b662-4561-a1e7-a92522a08819 None admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:13.312607 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 530/1069] 10.222.0.18 () {56 vars in 1093 bytes} [Wed Aug 24 17:09:13 2022] GET /identity/v3/users/nova => generated 82 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:09:13.315502 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec6c4f1-b662-4561-a1e7-a92522a08819 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:09:13.315891 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec6c4f1-b662-4561-a1e7-a92522a08819 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:13.317431 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec6c4f1-b662-4561-a1e7-a92522a08819 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:13.318000 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e176cb9-3b0e-4011-86d0-aaf6553d4bff None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:13.331607 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 540/1070] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 24 17:09:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:09:13.333072 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec6c4f1-b662-4561-a1e7-a92522a08819 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:13.394858 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:09:13.427344 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e176cb9-3b0e-4011-86d0-aaf6553d4bff None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:13.441696 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec6c4f1-b662-4561-a1e7-a92522a08819 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:13.452370 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:09:13.464860 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4e01a8-1b77-4293-ab78-af3560695ab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:13.465478 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4e01a8-1b77-4293-ab78-af3560695ab3 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:13.465753 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4e01a8-1b77-4293-ab78-af3560695ab3 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:13.466020 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4e01a8-1b77-4293-ab78-af3560695ab3 None admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:13.466754 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4e01a8-1b77-4293-ab78-af3560695ab3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:09:13.467101 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4e01a8-1b77-4293-ab78-af3560695ab3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:13.468522 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4e01a8-1b77-4293-ab78-af3560695ab3 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:13.481384 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0247f8bf-4601-4119-a469-cd7f0f2ac676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:13.482011 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0247f8bf-4601-4119-a469-cd7f0f2ac676 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:13.482286 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0247f8bf-4601-4119-a469-cd7f0f2ac676 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:13.482555 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0247f8bf-4601-4119-a469-cd7f0f2ac676 None admin] PATH_INFO: `/v3/users/e97038018a5e422a83f2235bd1240bf7` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:13.483835 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0247f8bf-4601-4119-a469-cd7f0f2ac676 None admin] RBAC: Authorizing `identity:get_user(user_id=e97038018a5e422a83f2235bd1240bf7)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:13.485238 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0247f8bf-4601-4119-a469-cd7f0f2ac676 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:13.486500 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 531/1071] 10.222.0.18 () {56 vars in 1133 bytes} [Wed Aug 24 17:09:13 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:09:13.491529 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 541/1072] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 24 17:09:13 2022] GET /identity/v3/users/e97038018a5e422a83f2235bd1240bf7 => generated 285 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:09:13.493809 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0247f8bf-4601-4119-a469-cd7f0f2ac676 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:13.493957 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4e01a8-1b77-4293-ab78-af3560695ab3 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:13.603768 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4e01a8-1b77-4293-ab78-af3560695ab3 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:13.606940 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0247f8bf-4601-4119-a469-cd7f0f2ac676 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:13.634311 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:09:13.640346 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c57b44-c2e5-4be8-b7ae-7a47aebeb70d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:13.640977 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c57b44-c2e5-4be8-b7ae-7a47aebeb70d None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:13.641262 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c57b44-c2e5-4be8-b7ae-7a47aebeb70d None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:13.652153 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c57b44-c2e5-4be8-b7ae-7a47aebeb70d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:13.652276 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07cd2ef-c4a8-4964-8c8a-828c6fccb35c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:13.652276 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07cd2ef-c4a8-4964-8c8a-828c6fccb35c None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:13.652276 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07cd2ef-c4a8-4964-8c8a-828c6fccb35c None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:13.652276 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07cd2ef-c4a8-4964-8c8a-828c6fccb35c None admin] PATH_INFO: `/v3/projects/408deae9dd0b48e79cc6b850a2f006b7` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:13.652553 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07cd2ef-c4a8-4964-8c8a-828c6fccb35c None admin] RBAC: Authorizing `identity:get_project(project_id=408deae9dd0b48e79cc6b850a2f006b7)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:13.652553 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07cd2ef-c4a8-4964-8c8a-828c6fccb35c None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:13.652678 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c57b44-c2e5-4be8-b7ae-7a47aebeb70d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:13.653670 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c57b44-c2e5-4be8-b7ae-7a47aebeb70d None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:13.658959 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 542/1073] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 24 17:09:13 2022] GET /identity/v3/projects/408deae9dd0b48e79cc6b850a2f006b7 => generated 311 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:09:13.659625 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07cd2ef-c4a8-4964-8c8a-828c6fccb35c None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:13.668767 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-c5c57b44-c2e5-4be8-b7ae-7a47aebeb70d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:09:13.668767 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:09:13.668767 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:09:13.668767 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:09:13.668767 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:09:13.668767 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:09:13.668767 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:09:13.668767 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:09:13.668767 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:09:13.668767 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:09:13.668767 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:09:13.668767 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:09:13.668767 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:09:13.668767 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:09:13.668767 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:09:13.668767 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:09:13.669716 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:09:13.669716 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:09:13.669716 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:09:13.669716 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:09:13.669716 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:09:13.669716 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:09:13.669716 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:09:13.669716 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 24 17:09:13.669716 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:09:13.669716 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:09:13.669716 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:09:13.669716 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:09:13.669716 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:09:13.669716 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:09:13.669716 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:09:13.669716 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:09:13.669716 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:09:13.670668 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:09:13.670668 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:09:13.670668 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:09:13.674849 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 532/1074] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 24 17:09:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:09:13.675692 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c57b44-c2e5-4be8-b7ae-7a47aebeb70d None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:13.713098 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:09:13.770293 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07cd2ef-c4a8-4964-8c8a-828c6fccb35c None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:13.784441 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c57b44-c2e5-4be8-b7ae-7a47aebeb70d None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:13.804418 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc57011f-1d12-4317-ae30-9bdbe2c34384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:13.805065 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc57011f-1d12-4317-ae30-9bdbe2c34384 None admin] REQUEST_METHOD: `PUT` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:13.805350 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc57011f-1d12-4317-ae30-9bdbe2c34384 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:13.805620 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc57011f-1d12-4317-ae30-9bdbe2c34384 None admin] PATH_INFO: `/v3/projects/408deae9dd0b48e79cc6b850a2f006b7/users/e97038018a5e422a83f2235bd1240bf7/roles/c5346bc54abf445f9900edf94264c850` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:13.807563 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc57011f-1d12-4317-ae30-9bdbe2c34384 None admin] RBAC: Authorizing `identity:create_grant(project_id=408deae9dd0b48e79cc6b850a2f006b7, user_id=e97038018a5e422a83f2235bd1240bf7, role_id=c5346bc54abf445f9900edf94264c850)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:13.808981 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc57011f-1d12-4317-ae30-9bdbe2c34384 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:13.820059 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8737061f-0c0c-4bcd-9960-b3e84b031adc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:13.820654 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8737061f-0c0c-4bcd-9960-b3e84b031adc None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:13.820930 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8737061f-0c0c-4bcd-9960-b3e84b031adc None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:13.821201 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8737061f-0c0c-4bcd-9960-b3e84b031adc None admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:13.821836 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8737061f-0c0c-4bcd-9960-b3e84b031adc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:09:13.822135 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8737061f-0c0c-4bcd-9960-b3e84b031adc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:13.824110 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8737061f-0c0c-4bcd-9960-b3e84b031adc None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:13.840947 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 533/1075] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 24 17:09:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:09:13.842331 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8737061f-0c0c-4bcd-9960-b3e84b031adc None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:13.874677 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 543/1076] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Aug 24 17:09:13 2022] PUT /identity/v3/projects/408deae9dd0b48e79cc6b850a2f006b7/users/e97038018a5e422a83f2235bd1240bf7/roles/c5346bc54abf445f9900edf94264c850 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:09:13.947764 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:09:13.955083 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8737061f-0c0c-4bcd-9960-b3e84b031adc None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:13.992430 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f55c05e-73f0-4ece-88bd-9a8999a483d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:13.993083 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f55c05e-73f0-4ece-88bd-9a8999a483d6 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:13.993360 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f55c05e-73f0-4ece-88bd-9a8999a483d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:13.993649 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f55c05e-73f0-4ece-88bd-9a8999a483d6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:14.002419 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f55c05e-73f0-4ece-88bd-9a8999a483d6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:14.004078 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f55c05e-73f0-4ece-88bd-9a8999a483d6 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:14.016712 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-3f55c05e-73f0-4ece-88bd-9a8999a483d6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:09:14.016712 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:09:14.016712 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:09:14.016712 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:09:14.016712 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:09:14.016712 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:09:14.016712 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:09:14.016712 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:09:14.016712 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:09:14.016712 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:09:14.016712 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:09:14.016712 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:09:14.016712 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:09:14.016712 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:09:14.016712 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:09:14.017678 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:09:14.017678 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:09:14.017678 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:09:14.017678 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:09:14.017678 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:09:14.017678 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:09:14.017678 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:09:14.017678 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:09:14.017678 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 24 17:09:14.017678 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:09:14.017678 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:09:14.017678 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:09:14.017678 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:09:14.017678 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:09:14.017678 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:09:14.017678 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:09:14.017678 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:09:14.018596 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:09:14.018596 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:09:14.018596 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:09:14.018596 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:09:14.018596 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:09:14.018596 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:09:14.018596 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:09:14.018596 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:09:14.018596 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 534/1077] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 24 17:09:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:09:14.023980 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc57011f-1d12-4317-ae30-9bdbe2c34384 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:14.082280 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:09:14.129224 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc57011f-1d12-4317-ae30-9bdbe2c34384 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:14.163026 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6d352a-623a-41c1-b138-732dbb916d7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:14.163631 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6d352a-623a-41c1-b138-732dbb916d7d None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:14.163906 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6d352a-623a-41c1-b138-732dbb916d7d None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:14.164173 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6d352a-623a-41c1-b138-732dbb916d7d None admin] PATH_INFO: `/v3/projects` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:14.164857 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6d352a-623a-41c1-b138-732dbb916d7d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:09:14.165158 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6d352a-623a-41c1-b138-732dbb916d7d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:14.166556 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6d352a-623a-41c1-b138-732dbb916d7d None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:14.182299 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 544/1078] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Aug 24 17:09:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:09:14.188716 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f55c05e-73f0-4ece-88bd-9a8999a483d6 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:14.288338 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f55c05e-73f0-4ece-88bd-9a8999a483d6 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:14.322386 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7f7ccf-cda4-46bc-bd0c-a1be1cc493a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:14.323082 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7f7ccf-cda4-46bc-bd0c-a1be1cc493a9 None admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:14.323953 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7f7ccf-cda4-46bc-bd0c-a1be1cc493a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:14.324227 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7f7ccf-cda4-46bc-bd0c-a1be1cc493a9 None admin] PATH_INFO: `/v3/projects/46ab158bc3644c888557781b0630b0b8/users/557aca9255cb4bbb8af52e796c5c5177/roles/967c5aaa9ba746a7acaa59f3e9691b31` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:14.326455 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7f7ccf-cda4-46bc-bd0c-a1be1cc493a9 None admin] RBAC: Authorizing `identity:create_grant(project_id=46ab158bc3644c888557781b0630b0b8, user_id=557aca9255cb4bbb8af52e796c5c5177, role_id=967c5aaa9ba746a7acaa59f3e9691b31)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:14.327910 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7f7ccf-cda4-46bc-bd0c-a1be1cc493a9 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:14.395250 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac47595-5b03-407c-8fd2-62b6d5eed812 None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:14.395544 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac47595-5b03-407c-8fd2-62b6d5eed812 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:14.395817 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac47595-5b03-407c-8fd2-62b6d5eed812 None None] PATH_INFO: `/` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:14.397249 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 545/1079] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:09:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 17:09:14.402250 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9844828-3183-4be3-90a3-736edbaca840 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:14.402532 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9844828-3183-4be3-90a3-736edbaca840 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:14.402803 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9844828-3183-4be3-90a3-736edbaca840 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:14.406871 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 535/1080] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Aug 24 17:09:14 2022] PUT /identity/v3/projects/46ab158bc3644c888557781b0630b0b8/users/557aca9255cb4bbb8af52e796c5c5177/roles/967c5aaa9ba746a7acaa59f3e9691b31 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:09:14.427004 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-c9844828-3183-4be3-90a3-736edbaca840 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:09:14.428331 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-c9844828-3183-4be3-90a3-736edbaca840 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:09:14.503391 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 546/1081] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:09:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 17:09:14.774251 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7f7ccf-cda4-46bc-bd0c-a1be1cc493a9 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:14.791373 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:09:14.880161 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7f7ccf-cda4-46bc-bd0c-a1be1cc493a9 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:14.914507 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fd3cd3-2687-457d-b5ee-bbdb2c8e8904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:14.915136 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fd3cd3-2687-457d-b5ee-bbdb2c8e8904 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:14.915412 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fd3cd3-2687-457d-b5ee-bbdb2c8e8904 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:14.915677 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fd3cd3-2687-457d-b5ee-bbdb2c8e8904 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:14.984559 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fd3cd3-2687-457d-b5ee-bbdb2c8e8904 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:14.986017 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fd3cd3-2687-457d-b5ee-bbdb2c8e8904 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:15.001482 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:09:15.090478 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 536/1082] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 24 17:09:14 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 17:09:15.092105 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9844828-3183-4be3-90a3-736edbaca840 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:15.133625 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:09:15.188619 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9844828-3183-4be3-90a3-736edbaca840 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:15.221301 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4b851a-d748-4894-b5bb-4b7220e9148a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:15.221983 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4b851a-d748-4894-b5bb-4b7220e9148a None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:15.222260 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4b851a-d748-4894-b5bb-4b7220e9148a None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:15.222527 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4b851a-d748-4894-b5bb-4b7220e9148a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:15.233215 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4b851a-d748-4894-b5bb-4b7220e9148a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:15.234639 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4b851a-d748-4894-b5bb-4b7220e9148a None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:15.249407 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-cd4b851a-d748-4894-b5bb-4b7220e9148a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:09:15.249407 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:09:15.249407 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:09:15.249407 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:09:15.249407 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:09:15.249407 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:09:15.249407 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:09:15.249407 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:09:15.249407 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:09:15.249407 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:09:15.249407 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:09:15.249407 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:09:15.249407 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:09:15.249407 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:09:15.249407 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:09:15.249407 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:09:15.250390 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:09:15.250390 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:09:15.250390 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:09:15.250390 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:09:15.250390 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:09:15.250390 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:09:15.250390 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:09:15.250390 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application with Lock( Aug 24 17:09:15.250390 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:09:15.250390 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:09:15.250390 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:09:15.250390 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:09:15.250390 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:09:15.250390 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:09:15.250390 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:09:15.250390 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:09:15.250390 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:09:15.256492 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:09:15.256492 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:09:15.256492 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:09:15.256492 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 547/1083] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 24 17:09:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:09:15.256732 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fd3cd3-2687-457d-b5ee-bbdb2c8e8904 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:15.358737 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fd3cd3-2687-457d-b5ee-bbdb2c8e8904 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:15.373848 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:09:15.394235 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74cf469-434b-48ff-9385-8b37c868b1fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:15.394788 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74cf469-434b-48ff-9385-8b37c868b1fa None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:15.395077 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74cf469-434b-48ff-9385-8b37c868b1fa None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:15.395346 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74cf469-434b-48ff-9385-8b37c868b1fa None admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:15.395990 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74cf469-434b-48ff-9385-8b37c868b1fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:09:15.396288 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74cf469-434b-48ff-9385-8b37c868b1fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:15.397651 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74cf469-434b-48ff-9385-8b37c868b1fa None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:15.415426 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 537/1084] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 24 17:09:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:09:15.828732 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e86af28-5abb-41e6-84a7-7eb748a48673 None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:15.829261 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e86af28-5abb-41e6-84a7-7eb748a48673 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:15.829261 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e86af28-5abb-41e6-84a7-7eb748a48673 None None] PATH_INFO: `/` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:15.830599 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 548/1085] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:09:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:09:15.835964 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2a1b05-59bd-40b2-9a88-53b4fc702fbf None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:15.836391 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2a1b05-59bd-40b2-9a88-53b4fc702fbf None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:15.836516 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2a1b05-59bd-40b2-9a88-53b4fc702fbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:15.854668 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-be2a1b05-59bd-40b2-9a88-53b4fc702fbf None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:09:15.855884 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-be2a1b05-59bd-40b2-9a88-53b4fc702fbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:09:15.923142 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 538/1086] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:09:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:09:16.192613 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e86af28-5abb-41e6-84a7-7eb748a48673 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:16.289405 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:09:16.297631 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e86af28-5abb-41e6-84a7-7eb748a48673 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:16.307642 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebb6edb-bf7b-40ad-a7e3-dccc7dfbca8c None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:16.307642 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebb6edb-bf7b-40ad-a7e3-dccc7dfbca8c None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:16.307642 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebb6edb-bf7b-40ad-a7e3-dccc7dfbca8c None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:16.308944 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 539/1087] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:09:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 17:09:16.313882 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25026870-6872-480c-9a04-0b3b75e23d96 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:16.314164 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25026870-6872-480c-9a04-0b3b75e23d96 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:16.314431 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25026870-6872-480c-9a04-0b3b75e23d96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:16.331708 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-25026870-6872-480c-9a04-0b3b75e23d96 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:09:16.332725 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee06700b-783c-49a3-8285-6f2ed2326aca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:16.332855 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-25026870-6872-480c-9a04-0b3b75e23d96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:09:16.333355 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee06700b-783c-49a3-8285-6f2ed2326aca None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:16.333632 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee06700b-783c-49a3-8285-6f2ed2326aca None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:16.333900 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee06700b-783c-49a3-8285-6f2ed2326aca None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:16.344349 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee06700b-783c-49a3-8285-6f2ed2326aca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:09:16.348991 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-ee06700b-783c-49a3-8285-6f2ed2326aca None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:09:16.348991 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:09:16.348991 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:09:16.348991 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:09:16.348991 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:09:16.348991 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:09:16.348991 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:09:16.348991 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:09:16.348991 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:09:16.348991 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:09:16.348991 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:09:16.348991 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:09:16.348991 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:09:16.348991 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:09:16.348991 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:09:16.348991 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:09:16.351505 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:09:16.351505 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:09:16.351505 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:09:16.351505 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:09:16.351505 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:09:16.351505 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:09:16.351505 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:09:16.351505 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:09:16.351505 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:09:16.351505 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:09:16.351505 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:09:16.351505 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application with Lock( Aug 24 17:09:16.351505 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:09:16.351505 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:09:16.351505 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:09:16.351505 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:09:16.352406 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:09:16.352406 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:09:16.352406 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:09:16.352406 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:09:16.352406 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:09:16.352406 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:09:16.352406 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:09:16.352406 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:09:16.352406 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:09:16.352406 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:09:16.352406 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:09:16.352406 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:09:16.352406 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 549/1088] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Aug 24 17:09:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:09:16.356669 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee06700b-783c-49a3-8285-6f2ed2326aca None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:16.400003 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 540/1089] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:09:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:09:16.452223 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee06700b-783c-49a3-8285-6f2ed2326aca None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:16.484589 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6dd9d2-a571-4306-a708-66e5bcf7e827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:16.485180 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6dd9d2-a571-4306-a708-66e5bcf7e827 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:16.485457 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6dd9d2-a571-4306-a708-66e5bcf7e827 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:16.485722 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6dd9d2-a571-4306-a708-66e5bcf7e827 None admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:16.486366 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6dd9d2-a571-4306-a708-66e5bcf7e827 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:09:16.486662 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6dd9d2-a571-4306-a708-66e5bcf7e827 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:16.488039 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6dd9d2-a571-4306-a708-66e5bcf7e827 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:16.496163 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 550/1090] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 24 17:09:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:09:16.502174 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25026870-6872-480c-9a04-0b3b75e23d96 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:16.535351 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:09:16.599663 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25026870-6872-480c-9a04-0b3b75e23d96 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:16.632488 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e62670-360f-4bfe-897b-3f3d26b020af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:16.633112 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e62670-360f-4bfe-897b-3f3d26b020af None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:16.633386 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e62670-360f-4bfe-897b-3f3d26b020af None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:16.633655 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e62670-360f-4bfe-897b-3f3d26b020af None admin] PATH_INFO: `/v3/users/e97038018a5e422a83f2235bd1240bf7` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:16.634741 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e62670-360f-4bfe-897b-3f3d26b020af None admin] RBAC: Authorizing `identity:get_user(user_id=e97038018a5e422a83f2235bd1240bf7)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:16.636678 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e62670-360f-4bfe-897b-3f3d26b020af None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:16.638236 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 541/1091] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 24 17:09:16 2022] GET /identity/v3/users/e97038018a5e422a83f2235bd1240bf7 => generated 285 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:09:16.644229 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6dd9d2-a571-4306-a708-66e5bcf7e827 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:16.662269 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:09:16.667857 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e62670-360f-4bfe-897b-3f3d26b020af None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:16.744250 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6dd9d2-a571-4306-a708-66e5bcf7e827 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:16.765817 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e62670-360f-4bfe-897b-3f3d26b020af None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:16.767348 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:09:16.778661 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed21e2c3-dfe2-4866-8b7a-aa2d57400c7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:16.779674 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed21e2c3-dfe2-4866-8b7a-aa2d57400c7a None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:16.779674 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed21e2c3-dfe2-4866-8b7a-aa2d57400c7a None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:16.779877 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed21e2c3-dfe2-4866-8b7a-aa2d57400c7a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:16.799506 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adea9386-b1e0-4cf2-b256-93e7c0d1b139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:16.800125 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adea9386-b1e0-4cf2-b256-93e7c0d1b139 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:16.800401 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adea9386-b1e0-4cf2-b256-93e7c0d1b139 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:16.800668 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adea9386-b1e0-4cf2-b256-93e7c0d1b139 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:16.810447 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adea9386-b1e0-4cf2-b256-93e7c0d1b139 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:09:16.815105 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-adea9386-b1e0-4cf2-b256-93e7c0d1b139 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 17:09:16.815105 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:09:16.815105 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:09:16.815105 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:09:16.815105 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:09:16.815105 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:09:16.815105 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:09:16.815105 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:09:16.815105 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:09:16.815105 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:09:16.815105 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:09:16.815105 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:09:16.815105 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:09:16.815105 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:09:16.815105 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:09:16.815105 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:09:16.816317 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:09:16.816317 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:09:16.816317 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:09:16.816317 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:09:16.816317 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:09:16.816317 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:09:16.816317 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:09:16.816317 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:09:16.816317 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:09:16.816317 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:09:16.816317 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:09:16.816317 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 24 17:09:16.816317 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:09:16.816317 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:09:16.816317 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:09:16.816317 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:09:16.817165 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:09:16.817165 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:09:16.817165 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:09:16.817165 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:09:16.817165 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:09:16.817165 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:09:16.817165 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:09:16.817165 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:09:16.817165 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:09:16.817165 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:09:16.817165 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 17:09:16.817165 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:09:16.817165 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 542/1092] 10.222.0.18 () {56 vars in 1120 bytes} [Wed Aug 24 17:09:16 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:09:16.821973 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adea9386-b1e0-4cf2-b256-93e7c0d1b139 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:16.847180 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed21e2c3-dfe2-4866-8b7a-aa2d57400c7a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:16.848621 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed21e2c3-dfe2-4866-8b7a-aa2d57400c7a None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:16.865152 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:09:16.923676 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adea9386-b1e0-4cf2-b256-93e7c0d1b139 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:16.945341 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 551/1093] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 24 17:09:16 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 17:09:16.951755 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed21e2c3-dfe2-4866-8b7a-aa2d57400c7a None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:16.956809 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd28cbd-ed5f-4daf-8bef-779886700b45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:16.957412 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd28cbd-ed5f-4daf-8bef-779886700b45 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:16.957692 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd28cbd-ed5f-4daf-8bef-779886700b45 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:16.957965 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd28cbd-ed5f-4daf-8bef-779886700b45 None admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:16.958627 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd28cbd-ed5f-4daf-8bef-779886700b45 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:09:16.958958 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd28cbd-ed5f-4daf-8bef-779886700b45 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:16.960322 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd28cbd-ed5f-4daf-8bef-779886700b45 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:16.969029 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 543/1094] 10.222.0.18 () {56 vars in 1115 bytes} [Wed Aug 24 17:09:16 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:09:16.974555 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd28cbd-ed5f-4daf-8bef-779886700b45 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:17.000278 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:09:17.052545 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed21e2c3-dfe2-4866-8b7a-aa2d57400c7a None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:17.069109 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:09:17.078430 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd28cbd-ed5f-4daf-8bef-779886700b45 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:17.088488 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045543ab-f54f-4d79-9734-a08a8ea71813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:17.089203 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045543ab-f54f-4d79-9734-a08a8ea71813 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:17.089487 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045543ab-f54f-4d79-9734-a08a8ea71813 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:17.089757 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045543ab-f54f-4d79-9734-a08a8ea71813 None admin] PATH_INFO: `/v3/projects/408deae9dd0b48e79cc6b850a2f006b7` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:17.090867 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045543ab-f54f-4d79-9734-a08a8ea71813 None admin] RBAC: Authorizing `identity:get_project(project_id=408deae9dd0b48e79cc6b850a2f006b7)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:17.092317 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045543ab-f54f-4d79-9734-a08a8ea71813 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:17.093956 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 552/1095] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 24 17:09:16 2022] GET /identity/v3/projects/408deae9dd0b48e79cc6b850a2f006b7 => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:09:17.100028 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045543ab-f54f-4d79-9734-a08a8ea71813 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:17.112908 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b624a3a-d850-412d-886c-995c9a7834ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:17.113570 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b624a3a-d850-412d-886c-995c9a7834ae None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:17.113866 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b624a3a-d850-412d-886c-995c9a7834ae None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:17.114130 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b624a3a-d850-412d-886c-995c9a7834ae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:17.125138 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b624a3a-d850-412d-886c-995c9a7834ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:17.126845 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b624a3a-d850-412d-886c-995c9a7834ae None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:17.141186 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-3b624a3a-d850-412d-886c-995c9a7834ae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:09:17.141186 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:09:17.141186 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:09:17.141186 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:09:17.141186 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:09:17.141186 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:09:17.141186 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:09:17.141186 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:09:17.141186 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:09:17.141186 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:09:17.141186 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:09:17.141186 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:09:17.141186 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:09:17.141186 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:09:17.141186 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:09:17.141186 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:09:17.142340 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:09:17.142340 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:09:17.142340 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:09:17.142340 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:09:17.142340 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:09:17.142340 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:09:17.142340 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:09:17.142340 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 24 17:09:17.142340 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:09:17.142340 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:09:17.142340 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:09:17.142340 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:09:17.142340 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:09:17.142340 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:09:17.142340 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:09:17.142340 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:09:17.142340 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:09:17.143815 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:09:17.143815 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:09:17.143815 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:09:17.143815 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 544/1096] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 24 17:09:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:09:17.148055 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b624a3a-d850-412d-886c-995c9a7834ae None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:17.198895 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045543ab-f54f-4d79-9734-a08a8ea71813 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:17.224130 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:09:17.232749 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8811af-fcf0-4799-975f-c0e272a9f363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:17.234973 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8811af-fcf0-4799-975f-c0e272a9f363 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:17.234973 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8811af-fcf0-4799-975f-c0e272a9f363 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:17.234973 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8811af-fcf0-4799-975f-c0e272a9f363 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:17.234973 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8811af-fcf0-4799-975f-c0e272a9f363 None admin] RBAC: Adding query filter params (role.id=c5346bc54abf445f9900edf94264c850, scope.project.id=408deae9dd0b48e79cc6b850a2f006b7, user.id=e97038018a5e422a83f2235bd1240bf7) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:09:17.234973 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8811af-fcf0-4799-975f-c0e272a9f363 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:17.236259 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8811af-fcf0-4799-975f-c0e272a9f363 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:17.241874 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 553/1097] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 24 17:09:17 2022] GET /identity/v3/role_assignments?user.id=e97038018a5e422a83f2235bd1240bf7&scope.project.id=408deae9dd0b48e79cc6b850a2f006b7&role.id=c5346bc54abf445f9900edf94264c850 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:09:17.246410 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b624a3a-d850-412d-886c-995c9a7834ae None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:17.283835 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91542f4e-5490-4d89-be75-9c22bde843a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:17.284371 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91542f4e-5490-4d89-be75-9c22bde843a3 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:17.284654 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91542f4e-5490-4d89-be75-9c22bde843a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:17.284929 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91542f4e-5490-4d89-be75-9c22bde843a3 None admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:17.285592 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91542f4e-5490-4d89-be75-9c22bde843a3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:09:17.285897 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91542f4e-5490-4d89-be75-9c22bde843a3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:17.287354 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91542f4e-5490-4d89-be75-9c22bde843a3 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:17.300682 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 545/1098] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 24 17:09:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:09:17.306554 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8811af-fcf0-4799-975f-c0e272a9f363 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:17.358488 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:09:17.412727 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8811af-fcf0-4799-975f-c0e272a9f363 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:17.446573 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbdcb0b-3e5b-47c1-84d1-45adf0602636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:17.447238 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbdcb0b-3e5b-47c1-84d1-45adf0602636 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:17.447518 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbdcb0b-3e5b-47c1-84d1-45adf0602636 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:17.447788 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbdcb0b-3e5b-47c1-84d1-45adf0602636 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:17.454837 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbdcb0b-3e5b-47c1-84d1-45adf0602636 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:17.456297 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbdcb0b-3e5b-47c1-84d1-45adf0602636 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:17.467593 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-afbdcb0b-3e5b-47c1-84d1-45adf0602636 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 17:09:17.467593 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:09:17.467593 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:09:17.467593 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:09:17.467593 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:09:17.467593 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:09:17.467593 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:09:17.467593 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:09:17.467593 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:09:17.467593 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:09:17.467593 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:09:17.467593 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:09:17.467593 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:09:17.467593 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:09:17.467593 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:09:17.467593 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:09:17.468702 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:09:17.468702 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:09:17.468702 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:09:17.468702 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:09:17.468702 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:09:17.468702 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:09:17.468702 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:09:17.468702 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:09:17.468702 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:09:17.468702 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:09:17.468702 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:09:17.468702 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application with Lock( Aug 24 17:09:17.468702 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:09:17.468702 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:09:17.468702 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:09:17.468702 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:09:17.468702 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:09:17.468702 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:09:17.470074 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:09:17.470074 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:09:17.470074 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:09:17.470074 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:09:17.470074 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:09:17.470074 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:09:17.470074 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:09:17.470074 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:09:17.470074 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 17:09:17.470074 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:09:17.470074 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 554/1099] 10.222.0.18 () {56 vars in 1093 bytes} [Wed Aug 24 17:09:17 2022] GET /identity/v3/users/nova => generated 82 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:09:17.474439 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91542f4e-5490-4d89-be75-9c22bde843a3 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:17.576780 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91542f4e-5490-4d89-be75-9c22bde843a3 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:17.610292 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a54ebd-9c1b-47c9-97a2-5f744d1e5742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:17.610887 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a54ebd-9c1b-47c9-97a2-5f744d1e5742 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:17.611195 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a54ebd-9c1b-47c9-97a2-5f744d1e5742 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:17.611465 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a54ebd-9c1b-47c9-97a2-5f744d1e5742 None admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:17.612185 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a54ebd-9c1b-47c9-97a2-5f744d1e5742 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:09:17.612488 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a54ebd-9c1b-47c9-97a2-5f744d1e5742 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:17.613900 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a54ebd-9c1b-47c9-97a2-5f744d1e5742 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:17.616435 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:09:17.631933 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 546/1100] 10.222.0.18 () {56 vars in 1133 bytes} [Wed Aug 24 17:09:17 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:09:17.637704 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbdcb0b-3e5b-47c1-84d1-45adf0602636 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:17.734398 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbdcb0b-3e5b-47c1-84d1-45adf0602636 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:17.743187 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:09:17.767549 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8410ec3c-4cfb-4fcf-bdc7-2633b6857d08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:17.768153 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8410ec3c-4cfb-4fcf-bdc7-2633b6857d08 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:17.768435 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8410ec3c-4cfb-4fcf-bdc7-2633b6857d08 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:17.768705 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8410ec3c-4cfb-4fcf-bdc7-2633b6857d08 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:17.836719 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8410ec3c-4cfb-4fcf-bdc7-2633b6857d08 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:17.838160 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8410ec3c-4cfb-4fcf-bdc7-2633b6857d08 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:17.939089 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 555/1101] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 24 17:09:17 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 17:09:17.944850 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a54ebd-9c1b-47c9-97a2-5f744d1e5742 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:18.040368 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a54ebd-9c1b-47c9-97a2-5f744d1e5742 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:18.073016 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9ef684-3116-4f44-9698-dae2faf3f047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:18.073641 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9ef684-3116-4f44-9698-dae2faf3f047 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:18.073922 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9ef684-3116-4f44-9698-dae2faf3f047 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:18.074194 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9ef684-3116-4f44-9698-dae2faf3f047 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:18.085264 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9ef684-3116-4f44-9698-dae2faf3f047 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:18.086698 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9ef684-3116-4f44-9698-dae2faf3f047 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:18.101250 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-fe9ef684-3116-4f44-9698-dae2faf3f047 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:09:18.101250 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:09:18.101250 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:09:18.101250 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:09:18.101250 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:09:18.101250 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:09:18.101250 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:09:18.101250 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:09:18.101250 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:09:18.101250 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:09:18.101250 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:09:18.101250 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:09:18.101250 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:09:18.101250 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:09:18.101250 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:09:18.101250 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:09:18.107536 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:09:18.107536 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:09:18.107536 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:09:18.107536 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:09:18.107536 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:09:18.107536 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:09:18.107536 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:09:18.107536 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 24 17:09:18.107536 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:09:18.107536 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:09:18.107536 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:09:18.107536 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:09:18.107536 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:09:18.107536 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:09:18.107536 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:09:18.107536 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:09:18.107536 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:09:18.108490 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:09:18.108490 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:09:18.108490 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:09:18.108490 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 547/1102] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 24 17:09:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:09:18.108737 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8410ec3c-4cfb-4fcf-bdc7-2633b6857d08 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:18.118497 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:09:18.204811 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8410ec3c-4cfb-4fcf-bdc7-2633b6857d08 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:18.239794 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf91340-a47a-4df3-b095-7b54be8705d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:18.240404 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf91340-a47a-4df3-b095-7b54be8705d6 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:18.240684 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf91340-a47a-4df3-b095-7b54be8705d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:18.240959 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf91340-a47a-4df3-b095-7b54be8705d6 None admin] PATH_INFO: `/v3/domains` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:18.241607 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf91340-a47a-4df3-b095-7b54be8705d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:09:18.241911 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf91340-a47a-4df3-b095-7b54be8705d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:18.243318 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf91340-a47a-4df3-b095-7b54be8705d6 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:18.255205 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 556/1103] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 24 17:09:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:09:18.261338 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9ef684-3116-4f44-9698-dae2faf3f047 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:18.307097 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:09:18.362117 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9ef684-3116-4f44-9698-dae2faf3f047 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:18.396108 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab24ca6b-cd87-44f4-9e60-22f1ad59f3eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:18.396733 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab24ca6b-cd87-44f4-9e60-22f1ad59f3eb None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:18.397014 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab24ca6b-cd87-44f4-9e60-22f1ad59f3eb None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:18.397288 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab24ca6b-cd87-44f4-9e60-22f1ad59f3eb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:18.404440 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab24ca6b-cd87-44f4-9e60-22f1ad59f3eb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:18.405879 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab24ca6b-cd87-44f4-9e60-22f1ad59f3eb None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:18.417352 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-ab24ca6b-cd87-44f4-9e60-22f1ad59f3eb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:09:18.417352 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:09:18.417352 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:09:18.417352 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:09:18.417352 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:09:18.417352 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:09:18.417352 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:09:18.417352 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:09:18.417352 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:09:18.417352 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:09:18.417352 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:09:18.417352 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:09:18.417352 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:09:18.417352 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:09:18.417352 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:09:18.418377 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:09:18.418377 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:09:18.418377 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:09:18.418377 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:09:18.418377 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:09:18.418377 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:09:18.418377 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:09:18.418377 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:09:18.418377 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 24 17:09:18.418377 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:09:18.418377 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:09:18.418377 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:09:18.418377 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:09:18.418377 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:09:18.418377 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:09:18.418377 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:09:18.418377 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:09:18.424401 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:09:18.424401 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:09:18.424401 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:09:18.424401 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:09:18.424401 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:09:18.424401 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:09:18.424401 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:09:18.424401 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:09:18.424401 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 548/1104] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 24 17:09:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:09:18.425092 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf91340-a47a-4df3-b095-7b54be8705d6 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:18.489075 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:09:18.523602 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf91340-a47a-4df3-b095-7b54be8705d6 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:18.558397 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ffb94b-d2a0-40c6-aeba-25b810370289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:18.559039 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ffb94b-d2a0-40c6-aeba-25b810370289 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:18.559320 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ffb94b-d2a0-40c6-aeba-25b810370289 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:18.559587 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ffb94b-d2a0-40c6-aeba-25b810370289 None admin] PATH_INFO: `/v3/projects` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:18.560273 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ffb94b-d2a0-40c6-aeba-25b810370289 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:09:18.560572 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ffb94b-d2a0-40c6-aeba-25b810370289 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:18.561973 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ffb94b-d2a0-40c6-aeba-25b810370289 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:18.574601 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 557/1105] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Aug 24 17:09:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:09:18.580560 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab24ca6b-cd87-44f4-9e60-22f1ad59f3eb None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:18.678677 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab24ca6b-cd87-44f4-9e60-22f1ad59f3eb None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:18.685856 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:09:18.713004 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c0a009-9a76-488a-b713-d2959927d725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:18.713610 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c0a009-9a76-488a-b713-d2959927d725 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:18.713917 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c0a009-9a76-488a-b713-d2959927d725 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:18.714192 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c0a009-9a76-488a-b713-d2959927d725 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:18.714930 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c0a009-9a76-488a-b713-d2959927d725 None admin] RBAC: Adding query filter params (role.id=967c5aaa9ba746a7acaa59f3e9691b31, scope.project.id=46ab158bc3644c888557781b0630b0b8, user.id=557aca9255cb4bbb8af52e796c5c5177) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:09:18.715301 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c0a009-9a76-488a-b713-d2959927d725 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:18.716768 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c0a009-9a76-488a-b713-d2959927d725 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:18.723277 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 549/1106] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 24 17:09:18 2022] GET /identity/v3/role_assignments?user.id=557aca9255cb4bbb8af52e796c5c5177&scope.project.id=46ab158bc3644c888557781b0630b0b8&role.id=967c5aaa9ba746a7acaa59f3e9691b31 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:09:19.180860 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0cfcaf-fdbe-4a9b-9bbb-1a736a85a7b6 None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:19.181385 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0cfcaf-fdbe-4a9b-9bbb-1a736a85a7b6 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:19.181385 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0cfcaf-fdbe-4a9b-9bbb-1a736a85a7b6 None None] PATH_INFO: `/` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:19.183365 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 558/1107] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:09:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:09:19.189034 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298dc5a9-b9cc-4a8b-a0ac-4331427ec53f None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:19.189468 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298dc5a9-b9cc-4a8b-a0ac-4331427ec53f None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:19.189717 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298dc5a9-b9cc-4a8b-a0ac-4331427ec53f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:19.208257 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-298dc5a9-b9cc-4a8b-a0ac-4331427ec53f None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:09:19.209461 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-298dc5a9-b9cc-4a8b-a0ac-4331427ec53f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:09:19.277299 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 550/1108] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:09:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:09:19.555199 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0cfcaf-fdbe-4a9b-9bbb-1a736a85a7b6 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:19.653250 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0cfcaf-fdbe-4a9b-9bbb-1a736a85a7b6 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:19.678130 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:09:19.685541 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcccbc91-0582-4653-8155-32acaee78bca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:19.686158 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcccbc91-0582-4653-8155-32acaee78bca None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:19.686437 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcccbc91-0582-4653-8155-32acaee78bca None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:19.693452 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcccbc91-0582-4653-8155-32acaee78bca None admin] PATH_INFO: `/v3/domains/default` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:19.693452 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcccbc91-0582-4653-8155-32acaee78bca None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:19.693452 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcccbc91-0582-4653-8155-32acaee78bca None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:19.693452 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 559/1109] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 24 17:09:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:09:19.696361 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298dc5a9-b9cc-4a8b-a0ac-4331427ec53f None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:19.791526 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:09:19.794485 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298dc5a9-b9cc-4a8b-a0ac-4331427ec53f None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:19.827595 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f08aa9-7823-478a-a3e3-079ea4112e12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:19.828198 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f08aa9-7823-478a-a3e3-079ea4112e12 None admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:19.828478 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f08aa9-7823-478a-a3e3-079ea4112e12 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:19.828749 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f08aa9-7823-478a-a3e3-079ea4112e12 None admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:19.829386 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f08aa9-7823-478a-a3e3-079ea4112e12 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:19.830801 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f08aa9-7823-478a-a3e3-079ea4112e12 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:19.928709 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 551/1110] 10.222.0.18 () {60 vars in 1132 bytes} [Wed Aug 24 17:09:19 2022] POST /identity/v3/users => generated 286 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:09:21.837657 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c69013-9fb8-4af4-a0d1-a2e87da3f6b7 None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:21.838192 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c69013-9fb8-4af4-a0d1-a2e87da3f6b7 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:21.838192 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c69013-9fb8-4af4-a0d1-a2e87da3f6b7 None None] PATH_INFO: `/` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:21.840085 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 560/1111] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:09:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:09:21.845335 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f63951-2cea-4daf-8266-ed6462791925 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:21.845754 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f63951-2cea-4daf-8266-ed6462791925 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:21.845876 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f63951-2cea-4daf-8266-ed6462791925 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:21.864141 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-86f63951-2cea-4daf-8266-ed6462791925 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:09:21.865305 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-86f63951-2cea-4daf-8266-ed6462791925 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:09:21.929955 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 552/1112] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:09:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:09:22.195223 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c69013-9fb8-4af4-a0d1-a2e87da3f6b7 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:22.301567 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c69013-9fb8-4af4-a0d1-a2e87da3f6b7 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:22.334779 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91d98a1-9129-499d-b3e8-4b92ca2ebb8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:22.335428 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91d98a1-9129-499d-b3e8-4b92ca2ebb8b None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:22.335706 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91d98a1-9129-499d-b3e8-4b92ca2ebb8b None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:22.335976 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91d98a1-9129-499d-b3e8-4b92ca2ebb8b None admin] PATH_INFO: `/v3/roles/af19a7a2fae5422da5bd7daea4bbfccf` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:22.337347 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91d98a1-9129-499d-b3e8-4b92ca2ebb8b None admin] RBAC: Authorizing `identity:get_role(role_id=af19a7a2fae5422da5bd7daea4bbfccf)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:22.338701 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91d98a1-9129-499d-b3e8-4b92ca2ebb8b None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:22.340119 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 561/1113] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 24 17:09:22 2022] GET /identity/v3/roles/af19a7a2fae5422da5bd7daea4bbfccf => generated 223 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:09:22.346021 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f63951-2cea-4daf-8266-ed6462791925 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:22.426304 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:09:22.441352 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f63951-2cea-4daf-8266-ed6462791925 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:22.475600 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471acef8-3843-4f4a-99c6-d4e8d39f17a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:22.476167 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471acef8-3843-4f4a-99c6-d4e8d39f17a5 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:22.476446 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471acef8-3843-4f4a-99c6-d4e8d39f17a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:22.476719 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471acef8-3843-4f4a-99c6-d4e8d39f17a5 None admin] PATH_INFO: `/v3/users/ab415840e5914e1a801a7d68abed7a4a` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:22.489438 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471acef8-3843-4f4a-99c6-d4e8d39f17a5 None admin] RBAC: Authorizing `identity:get_user(user_id=ab415840e5914e1a801a7d68abed7a4a)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:22.490875 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471acef8-3843-4f4a-99c6-d4e8d39f17a5 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:22.492510 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 553/1114] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 24 17:09:22 2022] GET /identity/v3/users/ab415840e5914e1a801a7d68abed7a4a => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:09:22.498871 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91d98a1-9129-499d-b3e8-4b92ca2ebb8b None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:22.561582 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:09:22.597740 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91d98a1-9129-499d-b3e8-4b92ca2ebb8b None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:22.631666 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39baf580-ce64-4b60-a89b-2d56e1ad730a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:22.632287 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39baf580-ce64-4b60-a89b-2d56e1ad730a None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:22.632566 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39baf580-ce64-4b60-a89b-2d56e1ad730a None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:22.632888 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39baf580-ce64-4b60-a89b-2d56e1ad730a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:22.700455 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39baf580-ce64-4b60-a89b-2d56e1ad730a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:22.701889 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39baf580-ce64-4b60-a89b-2d56e1ad730a None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:22.759834 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:09:22.796158 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 562/1115] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 24 17:09:22 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 17:09:22.802119 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471acef8-3843-4f4a-99c6-d4e8d39f17a5 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:22.898206 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471acef8-3843-4f4a-99c6-d4e8d39f17a5 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:22.931400 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80eebd12-4902-4b19-a0c4-36ed964fd117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:22.932021 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eebd12-4902-4b19-a0c4-36ed964fd117 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:22.932296 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eebd12-4902-4b19-a0c4-36ed964fd117 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:22.932568 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eebd12-4902-4b19-a0c4-36ed964fd117 None admin] PATH_INFO: `/v3/projects/408deae9dd0b48e79cc6b850a2f006b7` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:22.934285 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80eebd12-4902-4b19-a0c4-36ed964fd117 None admin] RBAC: Authorizing `identity:get_project(project_id=408deae9dd0b48e79cc6b850a2f006b7)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:22.935730 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80eebd12-4902-4b19-a0c4-36ed964fd117 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:22.937327 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 554/1116] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 24 17:09:22 2022] GET /identity/v3/projects/408deae9dd0b48e79cc6b850a2f006b7 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:09:22.943355 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39baf580-ce64-4b60-a89b-2d56e1ad730a None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:23.045055 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39baf580-ce64-4b60-a89b-2d56e1ad730a None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:23.077791 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193a83cd-e2f3-4d04-a04f-69e88620d880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:23.078380 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193a83cd-e2f3-4d04-a04f-69e88620d880 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:23.078659 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193a83cd-e2f3-4d04-a04f-69e88620d880 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:23.079048 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193a83cd-e2f3-4d04-a04f-69e88620d880 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:23.079673 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193a83cd-e2f3-4d04-a04f-69e88620d880 None admin] RBAC: Adding query filter params (role.id=af19a7a2fae5422da5bd7daea4bbfccf, scope.project.id=408deae9dd0b48e79cc6b850a2f006b7, user.id=ab415840e5914e1a801a7d68abed7a4a) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:09:23.079979 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193a83cd-e2f3-4d04-a04f-69e88620d880 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:23.081391 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193a83cd-e2f3-4d04-a04f-69e88620d880 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:23.086503 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 563/1117] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 24 17:09:22 2022] GET /identity/v3/role_assignments?user.id=ab415840e5914e1a801a7d68abed7a4a&scope.project.id=408deae9dd0b48e79cc6b850a2f006b7&role.id=af19a7a2fae5422da5bd7daea4bbfccf => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:09:24.926423 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e161f8e1-14b5-425b-872d-f9faaff465af None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:24.927079 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e161f8e1-14b5-425b-872d-f9faaff465af None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:24.927079 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e161f8e1-14b5-425b-872d-f9faaff465af None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:24.928329 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 555/1118] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:09:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:09:24.933923 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b69302-cb77-4684-bf51-d43c4261c809 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:24.934304 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b69302-cb77-4684-bf51-d43c4261c809 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:24.934452 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b69302-cb77-4684-bf51-d43c4261c809 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:24.951738 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-f7b69302-cb77-4684-bf51-d43c4261c809 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:09:24.952901 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-f7b69302-cb77-4684-bf51-d43c4261c809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:09:24.974136 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:09:25.016720 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 564/1119] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:09:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:09:25.287591 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e161f8e1-14b5-425b-872d-f9faaff465af None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:25.302710 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:09:25.399120 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e161f8e1-14b5-425b-872d-f9faaff465af None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:25.439523 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3ed586-f052-4142-b727-7297376f89ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:25.440149 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3ed586-f052-4142-b727-7297376f89ee None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:25.440430 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3ed586-f052-4142-b727-7297376f89ee None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:25.440701 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3ed586-f052-4142-b727-7297376f89ee None admin] PATH_INFO: `/v3/roles/af19a7a2fae5422da5bd7daea4bbfccf` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:25.442084 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3ed586-f052-4142-b727-7297376f89ee None admin] RBAC: Authorizing `identity:get_role(role_id=af19a7a2fae5422da5bd7daea4bbfccf)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:25.443504 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3ed586-f052-4142-b727-7297376f89ee None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:25.444832 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 556/1120] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 24 17:09:25 2022] GET /identity/v3/roles/af19a7a2fae5422da5bd7daea4bbfccf => generated 223 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:09:25.450673 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b69302-cb77-4684-bf51-d43c4261c809 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:25.546683 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b69302-cb77-4684-bf51-d43c4261c809 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:25.579468 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdcdd1a-45cb-4992-9907-09ea34b1c106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:25.580090 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdcdd1a-45cb-4992-9907-09ea34b1c106 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:25.580369 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdcdd1a-45cb-4992-9907-09ea34b1c106 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:25.580643 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdcdd1a-45cb-4992-9907-09ea34b1c106 None admin] PATH_INFO: `/v3/users/ab415840e5914e1a801a7d68abed7a4a` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:25.581750 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdcdd1a-45cb-4992-9907-09ea34b1c106 None admin] RBAC: Authorizing `identity:get_user(user_id=ab415840e5914e1a801a7d68abed7a4a)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:25.583188 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdcdd1a-45cb-4992-9907-09ea34b1c106 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:25.584755 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 565/1121] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 24 17:09:25 2022] GET /identity/v3/users/ab415840e5914e1a801a7d68abed7a4a => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:09:25.591423 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3ed586-f052-4142-b727-7297376f89ee None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:25.670115 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:09:25.689648 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3ed586-f052-4142-b727-7297376f89ee None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:25.724954 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25544627-41b8-4315-b972-e26ca44d54ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:25.725574 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25544627-41b8-4315-b972-e26ca44d54ac None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:25.725850 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25544627-41b8-4315-b972-e26ca44d54ac None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:25.726118 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25544627-41b8-4315-b972-e26ca44d54ac None admin] PATH_INFO: `/v3/projects/408deae9dd0b48e79cc6b850a2f006b7` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:25.727244 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25544627-41b8-4315-b972-e26ca44d54ac None admin] RBAC: Authorizing `identity:get_project(project_id=408deae9dd0b48e79cc6b850a2f006b7)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:25.728655 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25544627-41b8-4315-b972-e26ca44d54ac None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:25.730294 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 557/1122] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 24 17:09:25 2022] GET /identity/v3/projects/408deae9dd0b48e79cc6b850a2f006b7 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:09:25.736305 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdcdd1a-45cb-4992-9907-09ea34b1c106 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:25.763160 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:09:25.833255 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdcdd1a-45cb-4992-9907-09ea34b1c106 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:25.865883 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadd995c-450a-45e6-9e7b-9c1e84d45b01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:25.866529 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadd995c-450a-45e6-9e7b-9c1e84d45b01 None admin] REQUEST_METHOD: `PUT` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:25.866808 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadd995c-450a-45e6-9e7b-9c1e84d45b01 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:25.867112 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadd995c-450a-45e6-9e7b-9c1e84d45b01 None admin] PATH_INFO: `/v3/projects/408deae9dd0b48e79cc6b850a2f006b7/users/ab415840e5914e1a801a7d68abed7a4a/roles/af19a7a2fae5422da5bd7daea4bbfccf` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:25.869226 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadd995c-450a-45e6-9e7b-9c1e84d45b01 None admin] RBAC: Authorizing `identity:create_grant(project_id=408deae9dd0b48e79cc6b850a2f006b7, user_id=ab415840e5914e1a801a7d68abed7a4a, role_id=af19a7a2fae5422da5bd7daea4bbfccf)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:25.870663 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadd995c-450a-45e6-9e7b-9c1e84d45b01 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:25.953898 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 566/1123] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Aug 24 17:09:25 2022] PUT /identity/v3/projects/408deae9dd0b48e79cc6b850a2f006b7/users/ab415840e5914e1a801a7d68abed7a4a/roles/af19a7a2fae5422da5bd7daea4bbfccf => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:09:27.821168 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70a365e-da6e-4690-8aa1-b198cee65311 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:27.821678 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70a365e-da6e-4690-8aa1-b198cee65311 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:27.821798 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70a365e-da6e-4690-8aa1-b198cee65311 None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:27.823070 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 558/1124] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:09:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:09:27.829348 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935e0201-2219-452d-b500-75929c4c5259 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:27.829772 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935e0201-2219-452d-b500-75929c4c5259 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:27.829890 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935e0201-2219-452d-b500-75929c4c5259 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:27.847450 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-935e0201-2219-452d-b500-75929c4c5259 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:09:27.848703 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-935e0201-2219-452d-b500-75929c4c5259 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:09:27.913031 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 567/1125] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:09:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:09:28.182104 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70a365e-da6e-4690-8aa1-b198cee65311 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:28.285661 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70a365e-da6e-4690-8aa1-b198cee65311 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:28.326583 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2038edd-6b3d-4e2b-bf0f-936d20382c23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:28.327234 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2038edd-6b3d-4e2b-bf0f-936d20382c23 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:28.327513 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2038edd-6b3d-4e2b-bf0f-936d20382c23 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:28.327781 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2038edd-6b3d-4e2b-bf0f-936d20382c23 None admin] PATH_INFO: `/v3/roles/af19a7a2fae5422da5bd7daea4bbfccf` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:28.329147 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2038edd-6b3d-4e2b-bf0f-936d20382c23 None admin] RBAC: Authorizing `identity:get_role(role_id=af19a7a2fae5422da5bd7daea4bbfccf)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:28.330498 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2038edd-6b3d-4e2b-bf0f-936d20382c23 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:28.331844 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 559/1126] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 24 17:09:28 2022] GET /identity/v3/roles/af19a7a2fae5422da5bd7daea4bbfccf => generated 223 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:09:28.337608 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935e0201-2219-452d-b500-75929c4c5259 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:28.356648 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:09:28.434406 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935e0201-2219-452d-b500-75929c4c5259 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:28.466617 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75c99c5-b583-4d74-a84f-2de210a9b9a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:28.467260 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75c99c5-b583-4d74-a84f-2de210a9b9a6 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:28.467538 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75c99c5-b583-4d74-a84f-2de210a9b9a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:28.467808 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75c99c5-b583-4d74-a84f-2de210a9b9a6 None admin] PATH_INFO: `/v3/users/ab415840e5914e1a801a7d68abed7a4a` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:28.468903 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75c99c5-b583-4d74-a84f-2de210a9b9a6 None admin] RBAC: Authorizing `identity:get_user(user_id=ab415840e5914e1a801a7d68abed7a4a)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:28.470315 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75c99c5-b583-4d74-a84f-2de210a9b9a6 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:28.471904 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 568/1127] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 24 17:09:28 2022] GET /identity/v3/users/ab415840e5914e1a801a7d68abed7a4a => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:09:28.477771 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2038edd-6b3d-4e2b-bf0f-936d20382c23 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:28.504465 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:09:28.587138 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2038edd-6b3d-4e2b-bf0f-936d20382c23 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:28.620329 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f96ab8e-270c-45ce-b220-b98aa8360e28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:28.620932 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f96ab8e-270c-45ce-b220-b98aa8360e28 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:28.621211 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f96ab8e-270c-45ce-b220-b98aa8360e28 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:28.621523 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f96ab8e-270c-45ce-b220-b98aa8360e28 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:28.686110 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f96ab8e-270c-45ce-b220-b98aa8360e28 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:28.687511 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f96ab8e-270c-45ce-b220-b98aa8360e28 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:28.703839 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:09:28.782831 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 560/1128] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 24 17:09:28 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 17:09:28.788720 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75c99c5-b583-4d74-a84f-2de210a9b9a6 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:28.870495 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:09:28.885307 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75c99c5-b583-4d74-a84f-2de210a9b9a6 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:28.919449 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee78a70-c622-4e52-bbde-47c090bc0187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:28.920073 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee78a70-c622-4e52-bbde-47c090bc0187 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:28.920349 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee78a70-c622-4e52-bbde-47c090bc0187 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:28.920617 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee78a70-c622-4e52-bbde-47c090bc0187 None admin] PATH_INFO: `/v3/projects/408deae9dd0b48e79cc6b850a2f006b7` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:28.921706 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee78a70-c622-4e52-bbde-47c090bc0187 None admin] RBAC: Authorizing `identity:get_project(project_id=408deae9dd0b48e79cc6b850a2f006b7)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:28.923191 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee78a70-c622-4e52-bbde-47c090bc0187 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:28.924787 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 569/1129] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 24 17:09:28 2022] GET /identity/v3/projects/408deae9dd0b48e79cc6b850a2f006b7 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:09:28.930992 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f96ab8e-270c-45ce-b220-b98aa8360e28 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:29.027521 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f96ab8e-270c-45ce-b220-b98aa8360e28 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:29.042839 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:09:29.059945 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ea26d6-3982-42b7-a8bc-679538ad1724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:29.071239 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ea26d6-3982-42b7-a8bc-679538ad1724 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:29.071239 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ea26d6-3982-42b7-a8bc-679538ad1724 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:29.071239 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ea26d6-3982-42b7-a8bc-679538ad1724 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:29.071239 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ea26d6-3982-42b7-a8bc-679538ad1724 None admin] RBAC: Adding query filter params (role.id=af19a7a2fae5422da5bd7daea4bbfccf, scope.project.id=408deae9dd0b48e79cc6b850a2f006b7, user.id=ab415840e5914e1a801a7d68abed7a4a) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:09:29.071239 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ea26d6-3982-42b7-a8bc-679538ad1724 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:29.071239 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ea26d6-3982-42b7-a8bc-679538ad1724 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:29.071779 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 561/1130] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 24 17:09:28 2022] GET /identity/v3/role_assignments?user.id=ab415840e5914e1a801a7d68abed7a4a&scope.project.id=408deae9dd0b48e79cc6b850a2f006b7&role.id=af19a7a2fae5422da5bd7daea4bbfccf => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:09:30.957965 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8c5354-23be-4219-8e01-59598317238a None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:30.958561 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8c5354-23be-4219-8e01-59598317238a None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:30.958561 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8c5354-23be-4219-8e01-59598317238a None None] PATH_INFO: `/` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:30.959898 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 570/1131] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:09:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:09:30.965534 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66009a3a-4d84-4cb8-9740-30a2946094e5 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:30.965928 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66009a3a-4d84-4cb8-9740-30a2946094e5 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:30.966077 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66009a3a-4d84-4cb8-9740-30a2946094e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:30.983599 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-66009a3a-4d84-4cb8-9740-30a2946094e5 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:09:30.984750 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-66009a3a-4d84-4cb8-9740-30a2946094e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:09:31.048104 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 562/1132] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:09:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:09:31.312891 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8c5354-23be-4219-8e01-59598317238a None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:31.409672 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8c5354-23be-4219-8e01-59598317238a None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:31.442540 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611a978d-2fcd-4e81-92cb-9ec0995753de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:31.445556 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611a978d-2fcd-4e81-92cb-9ec0995753de None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:31.445556 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611a978d-2fcd-4e81-92cb-9ec0995753de None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:31.445556 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611a978d-2fcd-4e81-92cb-9ec0995753de None admin] PATH_INFO: `/v3/domains/default` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:31.445556 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611a978d-2fcd-4e81-92cb-9ec0995753de None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:31.446266 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611a978d-2fcd-4e81-92cb-9ec0995753de None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:31.447854 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 571/1133] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 24 17:09:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:09:31.453612 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66009a3a-4d84-4cb8-9740-30a2946094e5 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:31.550430 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66009a3a-4d84-4cb8-9740-30a2946094e5 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:31.560962 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:09:31.582885 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60360fe8-3f13-4ca5-85ca-bc436b73029d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:31.583516 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60360fe8-3f13-4ca5-85ca-bc436b73029d None admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:31.583790 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60360fe8-3f13-4ca5-85ca-bc436b73029d None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:31.584057 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60360fe8-3f13-4ca5-85ca-bc436b73029d None admin] PATH_INFO: `/v3/projects` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:31.584686 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60360fe8-3f13-4ca5-85ca-bc436b73029d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:31.586107 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60360fe8-3f13-4ca5-85ca-bc436b73029d None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:31.709174 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 563/1134] 10.222.0.18 () {60 vars in 1141 bytes} [Wed Aug 24 17:09:31 2022] POST /identity/v3/projects => generated 311 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:09:33.603403 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532b4924-39ea-4f1c-95fe-38137ae317fa None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:33.603949 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532b4924-39ea-4f1c-95fe-38137ae317fa None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:33.603949 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532b4924-39ea-4f1c-95fe-38137ae317fa None None] PATH_INFO: `/` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:33.605305 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 572/1135] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:09:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:09:33.610599 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5e2e25-f4c8-4015-9df1-b0a2769898eb None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:33.611018 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5e2e25-f4c8-4015-9df1-b0a2769898eb None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:33.611191 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5e2e25-f4c8-4015-9df1-b0a2769898eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:33.629561 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-4c5e2e25-f4c8-4015-9df1-b0a2769898eb None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:09:33.630738 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-4c5e2e25-f4c8-4015-9df1-b0a2769898eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:09:33.695692 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 564/1136] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:09:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:09:33.961527 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532b4924-39ea-4f1c-95fe-38137ae317fa None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:33.983285 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:09:34.060226 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532b4924-39ea-4f1c-95fe-38137ae317fa None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:34.093089 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f1613e-a8bb-41d5-b989-05875a1fc0c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:34.093704 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f1613e-a8bb-41d5-b989-05875a1fc0c2 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:34.093985 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f1613e-a8bb-41d5-b989-05875a1fc0c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:34.094256 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f1613e-a8bb-41d5-b989-05875a1fc0c2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:34.095388 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f1613e-a8bb-41d5-b989-05875a1fc0c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:34.096799 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f1613e-a8bb-41d5-b989-05875a1fc0c2 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:34.098333 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 573/1137] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 24 17:09:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:09:34.103995 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5e2e25-f4c8-4015-9df1-b0a2769898eb None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:34.190681 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:09:34.199687 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5e2e25-f4c8-4015-9df1-b0a2769898eb None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:34.236181 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4819ed4-42e1-4daa-88bd-3d14c0803f83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:34.236181 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4819ed4-42e1-4daa-88bd-3d14c0803f83 None admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:34.236181 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4819ed4-42e1-4daa-88bd-3d14c0803f83 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:34.236181 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4819ed4-42e1-4daa-88bd-3d14c0803f83 None admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:34.236589 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4819ed4-42e1-4daa-88bd-3d14c0803f83 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:34.236589 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4819ed4-42e1-4daa-88bd-3d14c0803f83 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:34.302266 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 565/1138] 10.222.0.18 () {60 vars in 1132 bytes} [Wed Aug 24 17:09:34 2022] POST /identity/v3/users => generated 286 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:09:36.224790 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745f589a-6100-4b53-a4aa-8f49abd595bf None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:36.225343 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745f589a-6100-4b53-a4aa-8f49abd595bf None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:36.225343 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745f589a-6100-4b53-a4aa-8f49abd595bf None None] PATH_INFO: `/` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:36.226651 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 574/1139] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:09:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:09:36.232636 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2b6464-6133-41ab-b1b1-2105dc8e64fe None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:36.233024 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2b6464-6133-41ab-b1b1-2105dc8e64fe None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:36.233192 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2b6464-6133-41ab-b1b1-2105dc8e64fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:36.251627 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-3d2b6464-6133-41ab-b1b1-2105dc8e64fe None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:09:36.252805 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-3d2b6464-6133-41ab-b1b1-2105dc8e64fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:09:36.317032 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 566/1140] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:09:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:09:36.608282 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745f589a-6100-4b53-a4aa-8f49abd595bf None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:36.698441 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:09:36.711187 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745f589a-6100-4b53-a4aa-8f49abd595bf None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:36.749508 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71b164e-335a-4f17-ba7f-028306d7ec09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:36.750138 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71b164e-335a-4f17-ba7f-028306d7ec09 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:36.750419 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71b164e-335a-4f17-ba7f-028306d7ec09 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:36.750690 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71b164e-335a-4f17-ba7f-028306d7ec09 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:36.762779 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71b164e-335a-4f17-ba7f-028306d7ec09 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:09:36.767467 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-e71b164e-335a-4f17-ba7f-028306d7ec09 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:09:36.767467 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:09:36.767467 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:09:36.767467 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:09:36.767467 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:09:36.767467 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:09:36.767467 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:09:36.767467 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:09:36.767467 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:09:36.767467 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:09:36.767467 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:09:36.767467 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:09:36.767467 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:09:36.767467 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:09:36.767467 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:09:36.767467 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:09:36.768614 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:09:36.768614 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:09:36.768614 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:09:36.768614 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:09:36.768614 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:09:36.768614 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:09:36.768614 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:09:36.768614 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:09:36.768614 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:09:36.768614 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:09:36.768614 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:09:36.768614 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application with Lock( Aug 24 17:09:36.768614 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:09:36.768614 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:09:36.768614 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:09:36.768614 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:09:36.769700 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:09:36.769700 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:09:36.769700 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:09:36.769700 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:09:36.769700 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:09:36.769700 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:09:36.769700 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:09:36.769700 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:09:36.769700 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:09:36.769700 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:09:36.769700 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:09:36.769700 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:09:36.771056 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 575/1141] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Aug 24 17:09:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:09:36.774737 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2b6464-6133-41ab-b1b1-2105dc8e64fe None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:36.854092 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:09:36.875335 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2b6464-6133-41ab-b1b1-2105dc8e64fe None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:36.910773 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2839f81-5505-4765-ad9a-cc6c9fc6c54d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:36.911426 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2839f81-5505-4765-ad9a-cc6c9fc6c54d None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:36.911704 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2839f81-5505-4765-ad9a-cc6c9fc6c54d None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:36.912253 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2839f81-5505-4765-ad9a-cc6c9fc6c54d None admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:36.912964 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2839f81-5505-4765-ad9a-cc6c9fc6c54d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:09:36.913265 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2839f81-5505-4765-ad9a-cc6c9fc6c54d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:36.914643 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2839f81-5505-4765-ad9a-cc6c9fc6c54d None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:36.923283 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 567/1142] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 24 17:09:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:09:36.929269 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71b164e-335a-4f17-ba7f-028306d7ec09 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:37.028851 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71b164e-335a-4f17-ba7f-028306d7ec09 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:37.061176 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9249d5-019c-4be9-82e8-82ea8b4682ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:37.061793 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9249d5-019c-4be9-82e8-82ea8b4682ef None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:37.062075 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9249d5-019c-4be9-82e8-82ea8b4682ef None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:37.062352 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9249d5-019c-4be9-82e8-82ea8b4682ef None admin] PATH_INFO: `/v3/users/ec621a72a4364c968a9c71a6cc4397bf` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:37.075942 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9249d5-019c-4be9-82e8-82ea8b4682ef None admin] RBAC: Authorizing `identity:get_user(user_id=ec621a72a4364c968a9c71a6cc4397bf)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:37.077379 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9249d5-019c-4be9-82e8-82ea8b4682ef None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:37.079016 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 576/1143] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 24 17:09:36 2022] GET /identity/v3/users/ec621a72a4364c968a9c71a6cc4397bf => generated 286 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:09:37.086164 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2839f81-5505-4765-ad9a-cc6c9fc6c54d None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:37.188121 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2839f81-5505-4765-ad9a-cc6c9fc6c54d None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:37.221715 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d1b635-e6f7-4ab0-a60c-d28e190191b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:37.222543 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d1b635-e6f7-4ab0-a60c-d28e190191b9 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:37.222851 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d1b635-e6f7-4ab0-a60c-d28e190191b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:37.223160 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d1b635-e6f7-4ab0-a60c-d28e190191b9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:37.233805 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:09:37.290532 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d1b635-e6f7-4ab0-a60c-d28e190191b9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:37.292012 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d1b635-e6f7-4ab0-a60c-d28e190191b9 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:37.387583 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 568/1144] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 24 17:09:37 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 17:09:37.393568 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9249d5-019c-4be9-82e8-82ea8b4682ef None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:37.402223 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:09:37.494405 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9249d5-019c-4be9-82e8-82ea8b4682ef None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:37.527475 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01c832d-ffcc-4cea-83ea-473336522f23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:37.528089 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01c832d-ffcc-4cea-83ea-473336522f23 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:37.528362 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01c832d-ffcc-4cea-83ea-473336522f23 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:37.528627 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01c832d-ffcc-4cea-83ea-473336522f23 None admin] PATH_INFO: `/v3/projects/fe3f1debbb7d49768f768c36474968ec` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:37.529709 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01c832d-ffcc-4cea-83ea-473336522f23 None admin] RBAC: Authorizing `identity:get_project(project_id=fe3f1debbb7d49768f768c36474968ec)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:37.531152 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01c832d-ffcc-4cea-83ea-473336522f23 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:37.532751 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 577/1145] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 24 17:09:37 2022] GET /identity/v3/projects/fe3f1debbb7d49768f768c36474968ec => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:09:37.538871 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d1b635-e6f7-4ab0-a60c-d28e190191b9 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:37.579655 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:09:37.635902 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d1b635-e6f7-4ab0-a60c-d28e190191b9 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:37.668429 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ddc73d-e32b-40e1-ba49-661d3f506d89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:37.669022 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ddc73d-e32b-40e1-ba49-661d3f506d89 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:37.669300 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ddc73d-e32b-40e1-ba49-661d3f506d89 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:37.669571 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ddc73d-e32b-40e1-ba49-661d3f506d89 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:37.670286 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ddc73d-e32b-40e1-ba49-661d3f506d89 None admin] RBAC: Adding query filter params (role.id=c5346bc54abf445f9900edf94264c850, scope.project.id=fe3f1debbb7d49768f768c36474968ec, user.id=ec621a72a4364c968a9c71a6cc4397bf) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:09:37.670595 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ddc73d-e32b-40e1-ba49-661d3f506d89 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:37.672054 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ddc73d-e32b-40e1-ba49-661d3f506d89 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:37.677222 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 569/1146] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 24 17:09:37 2022] GET /identity/v3/role_assignments?user.id=ec621a72a4364c968a9c71a6cc4397bf&scope.project.id=fe3f1debbb7d49768f768c36474968ec&role.id=c5346bc54abf445f9900edf94264c850 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:09:39.545686 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219d1137-fa36-41c0-ac8b-9b3374f17fef None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:39.546236 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219d1137-fa36-41c0-ac8b-9b3374f17fef None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:39.546236 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219d1137-fa36-41c0-ac8b-9b3374f17fef None None] PATH_INFO: `/` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:39.547601 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 578/1147] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:09:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:09:39.552729 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dba8c1b-ce60-4a7c-8974-a2314c3bfee5 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:39.553147 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dba8c1b-ce60-4a7c-8974-a2314c3bfee5 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:39.553315 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dba8c1b-ce60-4a7c-8974-a2314c3bfee5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:39.570973 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-3dba8c1b-ce60-4a7c-8974-a2314c3bfee5 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:09:39.572155 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-3dba8c1b-ce60-4a7c-8974-a2314c3bfee5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:09:39.636776 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 570/1148] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:09:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:09:39.903712 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219d1137-fa36-41c0-ac8b-9b3374f17fef None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:39.978113 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:09:40.004398 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219d1137-fa36-41c0-ac8b-9b3374f17fef None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:40.036925 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c36aa7-7325-4775-a994-51f6587f0f0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:40.037552 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c36aa7-7325-4775-a994-51f6587f0f0c None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:40.037833 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c36aa7-7325-4775-a994-51f6587f0f0c None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:40.038108 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c36aa7-7325-4775-a994-51f6587f0f0c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:40.048352 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c36aa7-7325-4775-a994-51f6587f0f0c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:09:40.053003 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-d4c36aa7-7325-4775-a994-51f6587f0f0c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:09:40.053003 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:09:40.053003 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:09:40.053003 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:09:40.053003 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:09:40.053003 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:09:40.053003 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:09:40.053003 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:09:40.053003 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:09:40.053003 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:09:40.053003 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:09:40.053003 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:09:40.053003 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:09:40.053003 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:09:40.053003 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:09:40.053003 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:09:40.054120 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:09:40.054120 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:09:40.054120 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:09:40.054120 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:09:40.054120 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:09:40.054120 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:09:40.054120 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:09:40.054120 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:09:40.054120 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:09:40.054120 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:09:40.054120 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:09:40.054120 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application with Lock( Aug 24 17:09:40.054120 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:09:40.054120 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:09:40.054120 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:09:40.054120 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:09:40.055399 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:09:40.055399 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:09:40.055399 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:09:40.055399 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:09:40.055399 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:09:40.055399 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:09:40.055399 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:09:40.055399 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:09:40.055399 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:09:40.055399 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:09:40.055399 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:09:40.055399 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:09:40.055399 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 579/1149] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Aug 24 17:09:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:09:40.060672 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dba8c1b-ce60-4a7c-8974-a2314c3bfee5 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:40.092793 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:09:40.157252 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dba8c1b-ce60-4a7c-8974-a2314c3bfee5 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:40.191491 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5df4b8-e6c9-4d00-810b-4b57235e4892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:40.192085 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5df4b8-e6c9-4d00-810b-4b57235e4892 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:40.192367 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5df4b8-e6c9-4d00-810b-4b57235e4892 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:40.192637 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5df4b8-e6c9-4d00-810b-4b57235e4892 None admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:40.193298 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5df4b8-e6c9-4d00-810b-4b57235e4892 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:09:40.193600 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5df4b8-e6c9-4d00-810b-4b57235e4892 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:40.194983 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5df4b8-e6c9-4d00-810b-4b57235e4892 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:40.202731 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 571/1150] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 24 17:09:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:09:40.208493 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c36aa7-7325-4775-a994-51f6587f0f0c None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:40.304198 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c36aa7-7325-4775-a994-51f6587f0f0c None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:40.335886 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29e223b-61fc-450f-8423-b84dcecca727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:40.336492 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29e223b-61fc-450f-8423-b84dcecca727 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:40.336771 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29e223b-61fc-450f-8423-b84dcecca727 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:40.337041 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29e223b-61fc-450f-8423-b84dcecca727 None admin] PATH_INFO: `/v3/users/ec621a72a4364c968a9c71a6cc4397bf` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:40.338132 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29e223b-61fc-450f-8423-b84dcecca727 None admin] RBAC: Authorizing `identity:get_user(user_id=ec621a72a4364c968a9c71a6cc4397bf)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:40.340090 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29e223b-61fc-450f-8423-b84dcecca727 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:40.341619 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 580/1151] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 24 17:09:40 2022] GET /identity/v3/users/ec621a72a4364c968a9c71a6cc4397bf => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:09:40.347257 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5df4b8-e6c9-4d00-810b-4b57235e4892 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:40.444278 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:09:40.449741 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5df4b8-e6c9-4d00-810b-4b57235e4892 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:40.482687 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f632548f-5ca2-4dcb-899b-35b582db1906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:40.483336 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f632548f-5ca2-4dcb-899b-35b582db1906 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:40.483621 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f632548f-5ca2-4dcb-899b-35b582db1906 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:40.483934 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f632548f-5ca2-4dcb-899b-35b582db1906 None admin] PATH_INFO: `/v3/projects/fe3f1debbb7d49768f768c36474968ec` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:40.485073 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f632548f-5ca2-4dcb-899b-35b582db1906 None admin] RBAC: Authorizing `identity:get_project(project_id=fe3f1debbb7d49768f768c36474968ec)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:40.529627 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f632548f-5ca2-4dcb-899b-35b582db1906 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:40.529627 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 572/1152] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 24 17:09:40 2022] GET /identity/v3/projects/fe3f1debbb7d49768f768c36474968ec => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:09:40.530441 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29e223b-61fc-450f-8423-b84dcecca727 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:40.530441 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:09:40.607318 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29e223b-61fc-450f-8423-b84dcecca727 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:40.639790 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209e72ab-6a8d-487f-b21d-ab0967ff1135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:40.640435 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209e72ab-6a8d-487f-b21d-ab0967ff1135 None admin] REQUEST_METHOD: `PUT` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:40.640715 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209e72ab-6a8d-487f-b21d-ab0967ff1135 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:40.640986 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209e72ab-6a8d-487f-b21d-ab0967ff1135 None admin] PATH_INFO: `/v3/projects/fe3f1debbb7d49768f768c36474968ec/users/ec621a72a4364c968a9c71a6cc4397bf/roles/c5346bc54abf445f9900edf94264c850` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:40.642861 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209e72ab-6a8d-487f-b21d-ab0967ff1135 None admin] RBAC: Authorizing `identity:create_grant(project_id=fe3f1debbb7d49768f768c36474968ec, user_id=ec621a72a4364c968a9c71a6cc4397bf, role_id=c5346bc54abf445f9900edf94264c850)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:40.644306 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209e72ab-6a8d-487f-b21d-ab0967ff1135 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:40.725062 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 581/1153] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Aug 24 17:09:40 2022] PUT /identity/v3/projects/fe3f1debbb7d49768f768c36474968ec/users/ec621a72a4364c968a9c71a6cc4397bf/roles/c5346bc54abf445f9900edf94264c850 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:09:42.608933 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952cd4bb-dbc1-4530-ae03-5a42c93d2c39 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:42.609405 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952cd4bb-dbc1-4530-ae03-5a42c93d2c39 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:42.609513 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952cd4bb-dbc1-4530-ae03-5a42c93d2c39 None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:42.610825 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 573/1154] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:09:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:09:42.616209 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ef36f0-fe54-460e-9be7-c707e326c2a2 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:42.616603 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ef36f0-fe54-460e-9be7-c707e326c2a2 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:42.616749 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ef36f0-fe54-460e-9be7-c707e326c2a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:42.634393 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-a0ef36f0-fe54-460e-9be7-c707e326c2a2 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:09:42.635597 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-a0ef36f0-fe54-460e-9be7-c707e326c2a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:09:42.689160 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:09:42.699108 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 582/1155] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:09:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:09:42.974685 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952cd4bb-dbc1-4530-ae03-5a42c93d2c39 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:43.072680 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952cd4bb-dbc1-4530-ae03-5a42c93d2c39 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:43.105211 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f6bfbe-a667-4863-a6b2-e6c21ba8f8c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:43.105828 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f6bfbe-a667-4863-a6b2-e6c21ba8f8c5 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:43.106106 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f6bfbe-a667-4863-a6b2-e6c21ba8f8c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:43.106378 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f6bfbe-a667-4863-a6b2-e6c21ba8f8c5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:43.116484 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f6bfbe-a667-4863-a6b2-e6c21ba8f8c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:09:43.121144 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-90f6bfbe-a667-4863-a6b2-e6c21ba8f8c5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:09:43.121144 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:09:43.121144 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:09:43.121144 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:09:43.121144 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:09:43.121144 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:09:43.121144 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:09:43.121144 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:09:43.121144 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:09:43.121144 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:09:43.121144 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:09:43.121144 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:09:43.121144 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:09:43.121144 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:09:43.121144 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:09:43.121144 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:09:43.122181 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:09:43.122181 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:09:43.122181 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:09:43.122181 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:09:43.122181 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:09:43.122181 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:09:43.122181 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:09:43.122181 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:09:43.122181 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:09:43.122181 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:09:43.122181 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:09:43.122181 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 24 17:09:43.122181 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:09:43.122181 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:09:43.122181 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:09:43.122181 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:09:43.123161 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:09:43.123161 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:09:43.123161 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:09:43.123161 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:09:43.123161 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:09:43.123161 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:09:43.123161 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:09:43.123161 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:09:43.123161 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:09:43.123161 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:09:43.123161 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:09:43.123161 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:09:43.123161 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 574/1156] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Aug 24 17:09:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:09:43.128416 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ef36f0-fe54-460e-9be7-c707e326c2a2 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:43.228166 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ef36f0-fe54-460e-9be7-c707e326c2a2 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:43.261753 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177cf08d-61a3-4cf1-a3ab-4b91ee73329a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:43.262349 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177cf08d-61a3-4cf1-a3ab-4b91ee73329a None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:43.262631 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177cf08d-61a3-4cf1-a3ab-4b91ee73329a None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:43.262929 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177cf08d-61a3-4cf1-a3ab-4b91ee73329a None admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:43.263595 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177cf08d-61a3-4cf1-a3ab-4b91ee73329a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:09:43.263898 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177cf08d-61a3-4cf1-a3ab-4b91ee73329a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:43.265252 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177cf08d-61a3-4cf1-a3ab-4b91ee73329a None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:43.273025 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 583/1157] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 24 17:09:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:09:43.279005 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f6bfbe-a667-4863-a6b2-e6c21ba8f8c5 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:43.303730 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:09:43.375589 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f6bfbe-a667-4863-a6b2-e6c21ba8f8c5 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:43.408640 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de837cf-e57d-4ad0-bdd8-8c3db73482ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:43.409266 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de837cf-e57d-4ad0-bdd8-8c3db73482ed None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:43.409547 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de837cf-e57d-4ad0-bdd8-8c3db73482ed None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:43.409821 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de837cf-e57d-4ad0-bdd8-8c3db73482ed None admin] PATH_INFO: `/v3/users/ec621a72a4364c968a9c71a6cc4397bf` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:43.410957 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de837cf-e57d-4ad0-bdd8-8c3db73482ed None admin] RBAC: Authorizing `identity:get_user(user_id=ec621a72a4364c968a9c71a6cc4397bf)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:43.412373 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de837cf-e57d-4ad0-bdd8-8c3db73482ed None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:43.413950 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 575/1158] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 24 17:09:43 2022] GET /identity/v3/users/ec621a72a4364c968a9c71a6cc4397bf => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:09:43.419906 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177cf08d-61a3-4cf1-a3ab-4b91ee73329a None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:43.474292 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:09:43.518396 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177cf08d-61a3-4cf1-a3ab-4b91ee73329a None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:43.555477 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8d3bc0-e6cc-4b4e-9fe1-938f519a5d4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:43.556109 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8d3bc0-e6cc-4b4e-9fe1-938f519a5d4f None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:43.556388 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8d3bc0-e6cc-4b4e-9fe1-938f519a5d4f None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:43.556657 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8d3bc0-e6cc-4b4e-9fe1-938f519a5d4f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:43.631355 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8d3bc0-e6cc-4b4e-9fe1-938f519a5d4f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:43.632832 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8d3bc0-e6cc-4b4e-9fe1-938f519a5d4f None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:43.684217 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:09:43.726567 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 584/1159] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 24 17:09:43 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 17:09:43.733032 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de837cf-e57d-4ad0-bdd8-8c3db73482ed None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:43.824790 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:09:43.833312 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de837cf-e57d-4ad0-bdd8-8c3db73482ed None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:43.866647 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd57d967-cfbe-44c1-98fa-af3776efe54f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:43.867308 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd57d967-cfbe-44c1-98fa-af3776efe54f None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:43.867588 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd57d967-cfbe-44c1-98fa-af3776efe54f None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:43.867862 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd57d967-cfbe-44c1-98fa-af3776efe54f None admin] PATH_INFO: `/v3/projects/fe3f1debbb7d49768f768c36474968ec` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:43.868962 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd57d967-cfbe-44c1-98fa-af3776efe54f None admin] RBAC: Authorizing `identity:get_project(project_id=fe3f1debbb7d49768f768c36474968ec)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:43.871036 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd57d967-cfbe-44c1-98fa-af3776efe54f None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:43.872667 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 576/1160] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 24 17:09:43 2022] GET /identity/v3/projects/fe3f1debbb7d49768f768c36474968ec => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:09:43.879201 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8d3bc0-e6cc-4b4e-9fe1-938f519a5d4f None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:43.983728 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8d3bc0-e6cc-4b4e-9fe1-938f519a5d4f None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:44.016760 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2d716f-ce59-4d67-a1c8-82d1321b6c2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:44.017355 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2d716f-ce59-4d67-a1c8-82d1321b6c2c None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:44.017634 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2d716f-ce59-4d67-a1c8-82d1321b6c2c None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:44.017902 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2d716f-ce59-4d67-a1c8-82d1321b6c2c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:44.018611 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2d716f-ce59-4d67-a1c8-82d1321b6c2c None admin] RBAC: Adding query filter params (role.id=c5346bc54abf445f9900edf94264c850, scope.project.id=fe3f1debbb7d49768f768c36474968ec, user.id=ec621a72a4364c968a9c71a6cc4397bf) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:09:44.018929 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2d716f-ce59-4d67-a1c8-82d1321b6c2c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:44.020345 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2d716f-ce59-4d67-a1c8-82d1321b6c2c None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:44.025832 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 585/1161] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 24 17:09:43 2022] GET /identity/v3/role_assignments?user.id=ec621a72a4364c968a9c71a6cc4397bf&scope.project.id=fe3f1debbb7d49768f768c36474968ec&role.id=c5346bc54abf445f9900edf94264c850 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:09:45.924132 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcfafd8-5add-4f27-b3c0-1cf810bb8755 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:45.924675 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcfafd8-5add-4f27-b3c0-1cf810bb8755 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:45.924675 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcfafd8-5add-4f27-b3c0-1cf810bb8755 None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:45.926009 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 577/1162] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:09:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:09:45.931095 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f63718-8e7c-4944-ae44-3ffdd5ae9e68 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:45.931530 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f63718-8e7c-4944-ae44-3ffdd5ae9e68 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:45.931634 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f63718-8e7c-4944-ae44-3ffdd5ae9e68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:45.949623 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-79f63718-8e7c-4944-ae44-3ffdd5ae9e68 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:09:45.950786 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-79f63718-8e7c-4944-ae44-3ffdd5ae9e68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:09:45.963344 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:09:46.015058 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 586/1163] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:09:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:09:46.285190 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcfafd8-5add-4f27-b3c0-1cf810bb8755 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:46.383830 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcfafd8-5add-4f27-b3c0-1cf810bb8755 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:46.425978 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280122f4-50ec-47b1-af8e-5e332ed1b11c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:46.426577 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280122f4-50ec-47b1-af8e-5e332ed1b11c None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:46.426856 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280122f4-50ec-47b1-af8e-5e332ed1b11c None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:46.427160 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280122f4-50ec-47b1-af8e-5e332ed1b11c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:46.454779 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:09:46.492334 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280122f4-50ec-47b1-af8e-5e332ed1b11c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:46.493778 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280122f4-50ec-47b1-af8e-5e332ed1b11c None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:46.590614 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 578/1164] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 24 17:09:46 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 17:09:46.596519 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f63718-8e7c-4944-ae44-3ffdd5ae9e68 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:46.690781 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f63718-8e7c-4944-ae44-3ffdd5ae9e68 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:46.722529 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0f00e0-9dff-4d46-885f-7dbef5c22b08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:46.723170 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0f00e0-9dff-4d46-885f-7dbef5c22b08 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:46.723449 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0f00e0-9dff-4d46-885f-7dbef5c22b08 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:46.723719 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0f00e0-9dff-4d46-885f-7dbef5c22b08 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:46.731527 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0f00e0-9dff-4d46-885f-7dbef5c22b08 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:46.732948 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0f00e0-9dff-4d46-885f-7dbef5c22b08 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:46.743768 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-3a0f00e0-9dff-4d46-885f-7dbef5c22b08 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 24 17:09:46.743768 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:09:46.743768 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 24 17:09:46.743768 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 24 17:09:46.743768 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:09:46.743768 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:09:46.743768 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:09:46.743768 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:09:46.743768 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 24 17:09:46.743768 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:09:46.743768 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 17:09:46.743768 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:09:46.743768 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:09:46.743768 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:09:46.743768 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:09:46.743768 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:09:46.743768 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:09:46.745551 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:09:46.745551 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:09:46.745551 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:09:46.745551 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:09:46.745551 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:09:46.745551 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:09:46.745551 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:09:46.745551 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:09:46.745551 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:09:46.745551 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:09:46.745551 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:09:46.745551 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:09:46.745551 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:09:46.745551 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:09:46.745551 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:09:46.745551 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:09:46.746614 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:09:46.746614 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application with Lock( Aug 24 17:09:46.746614 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:09:46.746614 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:09:46.746614 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:09:46.746614 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:09:46.746614 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:09:46.746614 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:09:46.746614 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:09:46.746614 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:09:46.746614 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 24 17:09:46.746614 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:09:46.746614 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 24 17:09:46.746614 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:09:46.746614 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 587/1165] 10.222.0.18 () {56 vars in 1117 bytes} [Wed Aug 24 17:09:46 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:09:46.751540 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280122f4-50ec-47b1-af8e-5e332ed1b11c None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:46.807915 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:09:46.845651 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280122f4-50ec-47b1-af8e-5e332ed1b11c None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:46.880213 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b2b7c1-712a-47a5-b9da-716d896db4c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:46.880814 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b2b7c1-712a-47a5-b9da-716d896db4c2 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:46.881090 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b2b7c1-712a-47a5-b9da-716d896db4c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:46.881364 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b2b7c1-712a-47a5-b9da-716d896db4c2 None admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:46.881997 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b2b7c1-712a-47a5-b9da-716d896db4c2 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:09:46.882298 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b2b7c1-712a-47a5-b9da-716d896db4c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:46.883681 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b2b7c1-712a-47a5-b9da-716d896db4c2 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:46.890578 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 579/1166] 10.222.0.18 () {56 vars in 1115 bytes} [Wed Aug 24 17:09:46 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:09:48.784357 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7a1510-1150-42be-b6ba-840efdd5aada None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:48.784905 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7a1510-1150-42be-b6ba-840efdd5aada None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:48.784905 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7a1510-1150-42be-b6ba-840efdd5aada None None] PATH_INFO: `/` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:48.786241 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 588/1167] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:09:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:09:48.791475 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bef6550-527a-4c8c-be9e-97aaab75174d None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:48.791899 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bef6550-527a-4c8c-be9e-97aaab75174d None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:48.792010 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bef6550-527a-4c8c-be9e-97aaab75174d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:48.810022 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-8bef6550-527a-4c8c-be9e-97aaab75174d None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:09:48.811162 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-8bef6550-527a-4c8c-be9e-97aaab75174d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:09:48.875476 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 580/1168] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:09:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:09:49.142566 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7a1510-1150-42be-b6ba-840efdd5aada None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:49.154338 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:09:49.243061 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7a1510-1150-42be-b6ba-840efdd5aada None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:49.275490 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba00ae03-fddf-4ea0-a235-324f0d32ac16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:49.276617 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba00ae03-fddf-4ea0-a235-324f0d32ac16 None admin] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:49.276897 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba00ae03-fddf-4ea0-a235-324f0d32ac16 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:49.277166 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba00ae03-fddf-4ea0-a235-324f0d32ac16 None admin] PATH_INFO: `/v3/domains` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:49.277816 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba00ae03-fddf-4ea0-a235-324f0d32ac16 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:49.279194 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba00ae03-fddf-4ea0-a235-324f0d32ac16 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:49.346204 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 589/1169] 10.222.0.18 () {60 vars in 1138 bytes} [Wed Aug 24 17:09:49 2022] POST /identity/v3/domains => generated 267 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:09:51.260103 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3634286-364b-4e7e-857d-5b22ed232603 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:51.261167 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3634286-364b-4e7e-857d-5b22ed232603 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:51.261261 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3634286-364b-4e7e-857d-5b22ed232603 None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:51.262552 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 581/1170] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:09:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:09:51.268175 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ea4bf7-ef33-40ca-90e5-a9f6ae9eea0c None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:51.268599 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ea4bf7-ef33-40ca-90e5-a9f6ae9eea0c None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:51.268712 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ea4bf7-ef33-40ca-90e5-a9f6ae9eea0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:51.286933 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-86ea4bf7-ef33-40ca-90e5-a9f6ae9eea0c None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:09:51.288124 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-86ea4bf7-ef33-40ca-90e5-a9f6ae9eea0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:09:51.346800 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:09:51.351968 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 590/1171] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:09:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:09:51.622954 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3634286-364b-4e7e-857d-5b22ed232603 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:51.677711 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:09:51.722532 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3634286-364b-4e7e-857d-5b22ed232603 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:51.755486 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04032e33-5bd6-42ce-b254-1019fee5f1ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:51.756104 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04032e33-5bd6-42ce-b254-1019fee5f1ee None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:51.756383 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04032e33-5bd6-42ce-b254-1019fee5f1ee None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:51.756656 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04032e33-5bd6-42ce-b254-1019fee5f1ee None admin] PATH_INFO: `/v3/domains/d247ea69d42b403288b9423444226e77` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:51.766760 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04032e33-5bd6-42ce-b254-1019fee5f1ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=d247ea69d42b403288b9423444226e77)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:51.768237 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04032e33-5bd6-42ce-b254-1019fee5f1ee None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:51.769811 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 582/1172] 10.222.0.18 () {56 vars in 1183 bytes} [Wed Aug 24 17:09:51 2022] GET /identity/v3/domains/d247ea69d42b403288b9423444226e77 => generated 267 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:09:51.775693 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ea4bf7-ef33-40ca-90e5-a9f6ae9eea0c None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:51.877397 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ea4bf7-ef33-40ca-90e5-a9f6ae9eea0c None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:51.910462 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b35370-7ff7-4624-8acc-6bb5eef87842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:51.911090 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b35370-7ff7-4624-8acc-6bb5eef87842 None admin] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:51.911370 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b35370-7ff7-4624-8acc-6bb5eef87842 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:51.911639 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b35370-7ff7-4624-8acc-6bb5eef87842 None admin] PATH_INFO: `/v3/projects` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:51.912274 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b35370-7ff7-4624-8acc-6bb5eef87842 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:51.913688 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b35370-7ff7-4624-8acc-6bb5eef87842 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:51.971545 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 591/1173] 10.222.0.18 () {60 vars in 1141 bytes} [Wed Aug 24 17:09:51 2022] POST /identity/v3/projects => generated 361 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:09:53.893708 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c04226-8e53-4a7a-a43a-8879274226bd None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:53.894240 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c04226-8e53-4a7a-a43a-8879274226bd None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:53.894240 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c04226-8e53-4a7a-a43a-8879274226bd None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:53.895621 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 583/1174] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:09:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:09:53.901023 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7387306e-5e2f-48ea-aa54-e461428be493 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:53.901435 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7387306e-5e2f-48ea-aa54-e461428be493 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:53.901573 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7387306e-5e2f-48ea-aa54-e461428be493 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:53.919174 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-7387306e-5e2f-48ea-aa54-e461428be493 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:09:53.920339 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-7387306e-5e2f-48ea-aa54-e461428be493 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:09:53.964996 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:09:53.984902 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 592/1175] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:09:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:09:54.254964 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c04226-8e53-4a7a-a43a-8879274226bd None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:54.357987 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c04226-8e53-4a7a-a43a-8879274226bd None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:54.372221 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:09:54.391805 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d9e2b5-1662-4ac5-8e99-84934eaa3d57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:54.392441 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d9e2b5-1662-4ac5-8e99-84934eaa3d57 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:54.392724 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d9e2b5-1662-4ac5-8e99-84934eaa3d57 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:54.392998 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d9e2b5-1662-4ac5-8e99-84934eaa3d57 None admin] PATH_INFO: `/v3/domains/d247ea69d42b403288b9423444226e77` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:54.394111 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d9e2b5-1662-4ac5-8e99-84934eaa3d57 None admin] RBAC: Authorizing `identity:get_domain(domain_id=d247ea69d42b403288b9423444226e77)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:54.395577 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d9e2b5-1662-4ac5-8e99-84934eaa3d57 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:54.397403 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 584/1176] 10.222.0.18 () {56 vars in 1183 bytes} [Wed Aug 24 17:09:54 2022] GET /identity/v3/domains/d247ea69d42b403288b9423444226e77 => generated 267 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:09:54.403184 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7387306e-5e2f-48ea-aa54-e461428be493 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:54.504652 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7387306e-5e2f-48ea-aa54-e461428be493 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:54.537108 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f7e535-6516-4388-91d5-9db67b0f7396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:54.537642 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f7e535-6516-4388-91d5-9db67b0f7396 None admin] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:54.537919 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f7e535-6516-4388-91d5-9db67b0f7396 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:54.538188 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f7e535-6516-4388-91d5-9db67b0f7396 None admin] PATH_INFO: `/v3/users` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:54.538834 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f7e535-6516-4388-91d5-9db67b0f7396 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:54.540278 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f7e535-6516-4388-91d5-9db67b0f7396 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:54.619525 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 593/1177] 10.222.0.18 () {60 vars in 1132 bytes} [Wed Aug 24 17:09:54 2022] POST /identity/v3/users => generated 311 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:09:56.552077 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f18ad78-86b9-45c8-9913-45b9a0db3675 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:56.552477 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f18ad78-86b9-45c8-9913-45b9a0db3675 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:56.552629 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f18ad78-86b9-45c8-9913-45b9a0db3675 None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:56.554002 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 585/1178] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:09:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:09:56.560127 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0ca964-1f7e-4cc5-b5d7-38fad2f3cf2b None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:56.560546 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0ca964-1f7e-4cc5-b5d7-38fad2f3cf2b None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:56.560671 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0ca964-1f7e-4cc5-b5d7-38fad2f3cf2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:56.578287 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-3d0ca964-1f7e-4cc5-b5d7-38fad2f3cf2b None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:09:56.579493 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-3d0ca964-1f7e-4cc5-b5d7-38fad2f3cf2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:09:56.608311 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:09:56.645188 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 594/1179] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:09:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:09:56.915169 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f18ad78-86b9-45c8-9913-45b9a0db3675 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:57.013862 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f18ad78-86b9-45c8-9913-45b9a0db3675 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:57.046734 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647dfb5b-5c70-4391-9954-d3c9451f6cb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:57.047376 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647dfb5b-5c70-4391-9954-d3c9451f6cb1 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:57.047652 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647dfb5b-5c70-4391-9954-d3c9451f6cb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:57.047920 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647dfb5b-5c70-4391-9954-d3c9451f6cb1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:57.058073 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647dfb5b-5c70-4391-9954-d3c9451f6cb1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:09:57.062692 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-647dfb5b-5c70-4391-9954-d3c9451f6cb1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:09:57.062692 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:09:57.062692 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:09:57.062692 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:09:57.062692 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:09:57.062692 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:09:57.062692 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:09:57.062692 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:09:57.062692 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:09:57.062692 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:09:57.062692 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:09:57.062692 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:09:57.062692 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:09:57.062692 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:09:57.062692 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:09:57.062692 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:09:57.063791 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:09:57.063791 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:09:57.063791 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:09:57.063791 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:09:57.063791 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:09:57.063791 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:09:57.063791 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:09:57.063791 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:09:57.063791 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:09:57.063791 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:09:57.063791 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:09:57.063791 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 24 17:09:57.063791 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:09:57.063791 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:09:57.063791 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:09:57.063791 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:09:57.064835 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:09:57.064835 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:09:57.064835 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:09:57.064835 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:09:57.064835 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:09:57.064835 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:09:57.064835 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:09:57.064835 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:09:57.064835 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:09:57.064835 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:09:57.064835 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:09:57.064835 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:09:57.064835 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 586/1180] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Aug 24 17:09:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:09:57.069562 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0ca964-1f7e-4cc5-b5d7-38fad2f3cf2b None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:57.164573 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0ca964-1f7e-4cc5-b5d7-38fad2f3cf2b None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:57.196791 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3061f3-11ee-4421-afe0-2d47b58974dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:57.197327 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3061f3-11ee-4421-afe0-2d47b58974dc None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:57.198109 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3061f3-11ee-4421-afe0-2d47b58974dc None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:57.198381 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3061f3-11ee-4421-afe0-2d47b58974dc None admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:57.199061 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3061f3-11ee-4421-afe0-2d47b58974dc None admin] RBAC: Adding query filter params (name=admin) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:09:57.199363 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3061f3-11ee-4421-afe0-2d47b58974dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:57.200700 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3061f3-11ee-4421-afe0-2d47b58974dc None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:57.208303 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 595/1181] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 24 17:09:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:09:57.214383 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647dfb5b-5c70-4391-9954-d3c9451f6cb1 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:57.255600 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:09:57.309701 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647dfb5b-5c70-4391-9954-d3c9451f6cb1 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:57.342167 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b39cd3d-7aba-4650-9edb-7408cb011324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:57.342780 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b39cd3d-7aba-4650-9edb-7408cb011324 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:57.343088 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b39cd3d-7aba-4650-9edb-7408cb011324 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:57.343359 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b39cd3d-7aba-4650-9edb-7408cb011324 None admin] PATH_INFO: `/v3/users/cc50f7ddb6bc4a43be77cd9e32064022` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:57.355775 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b39cd3d-7aba-4650-9edb-7408cb011324 None admin] RBAC: Authorizing `identity:get_user(user_id=cc50f7ddb6bc4a43be77cd9e32064022)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:57.360713 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b39cd3d-7aba-4650-9edb-7408cb011324 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:57.360713 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 587/1182] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 24 17:09:57 2022] GET /identity/v3/users/cc50f7ddb6bc4a43be77cd9e32064022 => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:09:57.364556 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3061f3-11ee-4421-afe0-2d47b58974dc None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:57.393034 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:09:57.462242 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3061f3-11ee-4421-afe0-2d47b58974dc None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:57.495130 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9ecee9-9244-45a9-8ed7-9c7294e8bcbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:57.495727 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9ecee9-9244-45a9-8ed7-9c7294e8bcbc None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:57.496006 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9ecee9-9244-45a9-8ed7-9c7294e8bcbc None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:57.496275 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9ecee9-9244-45a9-8ed7-9c7294e8bcbc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:57.562224 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9ecee9-9244-45a9-8ed7-9c7294e8bcbc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:57.563700 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9ecee9-9244-45a9-8ed7-9c7294e8bcbc None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:57.589821 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:09:57.659372 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 596/1183] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 24 17:09:57 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 17:09:57.665629 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b39cd3d-7aba-4650-9edb-7408cb011324 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:57.760013 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:09:57.763500 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b39cd3d-7aba-4650-9edb-7408cb011324 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:57.795977 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad45d49-b2a6-4701-a4ac-87425f8b1c31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:57.796587 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad45d49-b2a6-4701-a4ac-87425f8b1c31 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:57.796864 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad45d49-b2a6-4701-a4ac-87425f8b1c31 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:57.797135 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad45d49-b2a6-4701-a4ac-87425f8b1c31 None admin] PATH_INFO: `/v3/projects/c3de589d8a8b48e79e99730bb74a496f` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:57.798222 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad45d49-b2a6-4701-a4ac-87425f8b1c31 None admin] RBAC: Authorizing `identity:get_project(project_id=c3de589d8a8b48e79e99730bb74a496f)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:57.799653 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad45d49-b2a6-4701-a4ac-87425f8b1c31 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:57.801251 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 588/1184] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 24 17:09:57 2022] GET /identity/v3/projects/c3de589d8a8b48e79e99730bb74a496f => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:09:57.808154 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9ecee9-9244-45a9-8ed7-9c7294e8bcbc None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:09:57.902661 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9ecee9-9244-45a9-8ed7-9c7294e8bcbc None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:09:57.927857 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:09:57.935855 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debcbaa9-d21a-4038-aaaa-acd1d01b6424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:09:57.939903 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debcbaa9-d21a-4038-aaaa-acd1d01b6424 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:57.939903 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debcbaa9-d21a-4038-aaaa-acd1d01b6424 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:57.939903 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debcbaa9-d21a-4038-aaaa-acd1d01b6424 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:57.939903 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debcbaa9-d21a-4038-aaaa-acd1d01b6424 None admin] RBAC: Adding query filter params (role.id=c5346bc54abf445f9900edf94264c850, scope.project.id=c3de589d8a8b48e79e99730bb74a496f, user.id=cc50f7ddb6bc4a43be77cd9e32064022) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:09:57.939903 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debcbaa9-d21a-4038-aaaa-acd1d01b6424 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:09:57.939903 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debcbaa9-d21a-4038-aaaa-acd1d01b6424 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:09:57.944434 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 597/1185] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 24 17:09:57 2022] GET /identity/v3/role_assignments?user.id=cc50f7ddb6bc4a43be77cd9e32064022&scope.project.id=c3de589d8a8b48e79e99730bb74a496f&role.id=c5346bc54abf445f9900edf94264c850 => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:09:59.831162 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ecd9f7-af9c-49a6-b7de-f71c36c599cf None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:59.831669 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ecd9f7-af9c-49a6-b7de-f71c36c599cf None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:59.831760 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ecd9f7-af9c-49a6-b7de-f71c36c599cf None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:59.833032 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 589/1186] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:09:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:09:59.838089 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150af54b-6ea2-4432-acee-82cac7766577 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:09:59.839015 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150af54b-6ea2-4432-acee-82cac7766577 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:09:59.839197 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150af54b-6ea2-4432-acee-82cac7766577 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:09:59.857269 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-150af54b-6ea2-4432-acee-82cac7766577 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:09:59.858453 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-150af54b-6ea2-4432-acee-82cac7766577 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:09:59.922855 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 598/1187] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:09:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:10:00.191414 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ecd9f7-af9c-49a6-b7de-f71c36c599cf None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:10:00.289481 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ecd9f7-af9c-49a6-b7de-f71c36c599cf None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:10:00.321453 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccc7a8c-5e02-436b-bfd2-59e09645b8f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:10:00.322066 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccc7a8c-5e02-436b-bfd2-59e09645b8f7 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:10:00.322345 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccc7a8c-5e02-436b-bfd2-59e09645b8f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:10:00.323148 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccc7a8c-5e02-436b-bfd2-59e09645b8f7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:10:00.332896 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccc7a8c-5e02-436b-bfd2-59e09645b8f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:10:00.337570 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-3ccc7a8c-5e02-436b-bfd2-59e09645b8f7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:10:00.337570 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:10:00.337570 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:10:00.337570 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:10:00.337570 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:10:00.337570 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:10:00.337570 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:10:00.337570 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:10:00.337570 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:10:00.337570 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:10:00.337570 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:10:00.337570 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:10:00.337570 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:10:00.337570 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:10:00.337570 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:10:00.337570 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:10:00.339262 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:10:00.339262 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:10:00.339262 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:10:00.339262 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:10:00.339262 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:10:00.339262 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:10:00.339262 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:10:00.339262 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:10:00.339262 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:10:00.339262 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:10:00.339262 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:10:00.339262 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 24 17:10:00.339262 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:10:00.339262 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:10:00.339262 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:10:00.339262 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:10:00.340400 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:10:00.340400 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:10:00.340400 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:10:00.340400 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:10:00.340400 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:10:00.340400 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:10:00.340400 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:10:00.340400 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:10:00.340400 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:10:00.340400 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:10:00.340400 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:10:00.340400 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:10:00.340400 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 590/1188] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Aug 24 17:10:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:10:00.345223 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150af54b-6ea2-4432-acee-82cac7766577 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:10:00.439816 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150af54b-6ea2-4432-acee-82cac7766577 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:10:00.461725 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:10:00.473225 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a695ade0-99e7-48a9-b43a-fd3aab25d8b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:10:00.473823 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a695ade0-99e7-48a9-b43a-fd3aab25d8b3 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:10:00.474103 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a695ade0-99e7-48a9-b43a-fd3aab25d8b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:10:00.474374 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a695ade0-99e7-48a9-b43a-fd3aab25d8b3 None admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:10:00.475066 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a695ade0-99e7-48a9-b43a-fd3aab25d8b3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:10:00.475368 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a695ade0-99e7-48a9-b43a-fd3aab25d8b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:10:00.476713 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a695ade0-99e7-48a9-b43a-fd3aab25d8b3 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:10:00.484378 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 599/1189] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 24 17:10:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:10:00.490398 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccc7a8c-5e02-436b-bfd2-59e09645b8f7 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:10:00.519477 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:10:00.590144 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccc7a8c-5e02-436b-bfd2-59e09645b8f7 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:10:00.624029 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b624c1d9-a664-4fac-abca-a7cb3179ae1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:10:00.624648 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b624c1d9-a664-4fac-abca-a7cb3179ae1b None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:10:00.624931 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b624c1d9-a664-4fac-abca-a7cb3179ae1b None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:10:00.625206 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b624c1d9-a664-4fac-abca-a7cb3179ae1b None admin] PATH_INFO: `/v3/users/cc50f7ddb6bc4a43be77cd9e32064022` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:10:00.626579 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b624c1d9-a664-4fac-abca-a7cb3179ae1b None admin] RBAC: Authorizing `identity:get_user(user_id=cc50f7ddb6bc4a43be77cd9e32064022)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:10:00.628415 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b624c1d9-a664-4fac-abca-a7cb3179ae1b None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:10:00.630104 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 591/1190] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 24 17:10:00 2022] GET /identity/v3/users/cc50f7ddb6bc4a43be77cd9e32064022 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:10:00.636689 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a695ade0-99e7-48a9-b43a-fd3aab25d8b3 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:10:00.742070 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a695ade0-99e7-48a9-b43a-fd3aab25d8b3 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:10:00.775073 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7998156d-10b0-4718-92eb-01140e950f0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:10:00.775718 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7998156d-10b0-4718-92eb-01140e950f0b None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:10:00.776001 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7998156d-10b0-4718-92eb-01140e950f0b None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:10:00.776270 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7998156d-10b0-4718-92eb-01140e950f0b None admin] PATH_INFO: `/v3/projects/c3de589d8a8b48e79e99730bb74a496f` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:10:00.777373 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7998156d-10b0-4718-92eb-01140e950f0b None admin] RBAC: Authorizing `identity:get_project(project_id=c3de589d8a8b48e79e99730bb74a496f)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:10:00.778783 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7998156d-10b0-4718-92eb-01140e950f0b None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:10:00.780431 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 600/1191] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 24 17:10:00 2022] GET /identity/v3/projects/c3de589d8a8b48e79e99730bb74a496f => generated 361 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:10:00.786514 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b624c1d9-a664-4fac-abca-a7cb3179ae1b None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:10:00.878611 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:10:00.884134 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b624c1d9-a664-4fac-abca-a7cb3179ae1b None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:10:00.916726 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc41f8d-d8a8-4ce0-a4e1-9638d501f9a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:10:00.917407 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc41f8d-d8a8-4ce0-a4e1-9638d501f9a3 None admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:10:00.917683 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc41f8d-d8a8-4ce0-a4e1-9638d501f9a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:10:00.917951 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc41f8d-d8a8-4ce0-a4e1-9638d501f9a3 None admin] PATH_INFO: `/v3/projects/c3de589d8a8b48e79e99730bb74a496f/users/cc50f7ddb6bc4a43be77cd9e32064022/roles/c5346bc54abf445f9900edf94264c850` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:10:00.919839 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc41f8d-d8a8-4ce0-a4e1-9638d501f9a3 None admin] RBAC: Authorizing `identity:create_grant(project_id=c3de589d8a8b48e79e99730bb74a496f, user_id=cc50f7ddb6bc4a43be77cd9e32064022, role_id=c5346bc54abf445f9900edf94264c850)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:10:00.921261 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc41f8d-d8a8-4ce0-a4e1-9638d501f9a3 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:10:00.984433 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 592/1192] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Aug 24 17:10:00 2022] PUT /identity/v3/projects/c3de589d8a8b48e79e99730bb74a496f/users/cc50f7ddb6bc4a43be77cd9e32064022/roles/c5346bc54abf445f9900edf94264c850 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:10:02.878150 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afb6ec7-86e5-4631-bc51-b91aff7739b6 None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:10:02.878691 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afb6ec7-86e5-4631-bc51-b91aff7739b6 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:10:02.878691 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afb6ec7-86e5-4631-bc51-b91aff7739b6 None None] PATH_INFO: `/` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:10:02.880078 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 601/1193] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:10:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:10:02.885575 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ba5beb-4fc7-4330-a330-04723f82341d None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:10:02.885953 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ba5beb-4fc7-4330-a330-04723f82341d None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:10:02.886125 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ba5beb-4fc7-4330-a330-04723f82341d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:10:02.904149 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-22ba5beb-4fc7-4330-a330-04723f82341d None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:10:02.905313 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-22ba5beb-4fc7-4330-a330-04723f82341d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:10:02.969132 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 593/1194] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:10:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:10:03.235187 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afb6ec7-86e5-4631-bc51-b91aff7739b6 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:10:03.291776 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:10:03.338869 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afb6ec7-86e5-4631-bc51-b91aff7739b6 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:10:03.370890 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fba8c89-ef1b-4278-85ca-ab53c4205db2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:10:03.371539 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fba8c89-ef1b-4278-85ca-ab53c4205db2 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:10:03.371821 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fba8c89-ef1b-4278-85ca-ab53c4205db2 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:10:03.372142 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fba8c89-ef1b-4278-85ca-ab53c4205db2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:10:03.382760 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fba8c89-ef1b-4278-85ca-ab53c4205db2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:10:03.388067 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-3fba8c89-ef1b-4278-85ca-ab53c4205db2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:10:03.388067 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:10:03.388067 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:10:03.388067 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:10:03.388067 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:10:03.388067 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:10:03.388067 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:10:03.388067 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:10:03.388067 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:10:03.388067 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:10:03.388067 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:10:03.388067 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:10:03.388067 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:10:03.388067 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:10:03.388067 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:10:03.388067 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:10:03.389241 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:10:03.389241 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:10:03.389241 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:10:03.389241 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:10:03.389241 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:10:03.389241 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:10:03.389241 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:10:03.389241 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:10:03.389241 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:10:03.389241 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:10:03.389241 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:10:03.389241 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application with Lock( Aug 24 17:10:03.389241 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:10:03.389241 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:10:03.389241 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:10:03.389241 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:10:03.390318 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:10:03.390318 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:10:03.390318 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:10:03.390318 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:10:03.390318 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:10:03.390318 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:10:03.390318 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:10:03.390318 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:10:03.390318 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:10:03.390318 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:10:03.390318 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:10:03.390318 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:10:03.390318 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 602/1195] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Aug 24 17:10:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:10:03.395258 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ba5beb-4fc7-4330-a330-04723f82341d None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:10:03.477551 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:10:03.491752 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ba5beb-4fc7-4330-a330-04723f82341d None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:10:03.525421 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253f7ab0-8feb-4b2f-9f5a-48e67f72c350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:10:03.526059 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253f7ab0-8feb-4b2f-9f5a-48e67f72c350 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:10:03.526329 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253f7ab0-8feb-4b2f-9f5a-48e67f72c350 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:10:03.526596 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253f7ab0-8feb-4b2f-9f5a-48e67f72c350 None admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:10:03.527275 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253f7ab0-8feb-4b2f-9f5a-48e67f72c350 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:10:03.527574 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253f7ab0-8feb-4b2f-9f5a-48e67f72c350 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:10:03.528918 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253f7ab0-8feb-4b2f-9f5a-48e67f72c350 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:10:03.536895 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 594/1196] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 24 17:10:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:10:03.542376 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fba8c89-ef1b-4278-85ca-ab53c4205db2 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:10:03.640106 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fba8c89-ef1b-4278-85ca-ab53c4205db2 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:10:03.652125 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:10:03.672528 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63265708-3e82-439d-9c5b-420d78b88173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:10:03.678775 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63265708-3e82-439d-9c5b-420d78b88173 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:10:03.679088 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63265708-3e82-439d-9c5b-420d78b88173 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:10:03.679368 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63265708-3e82-439d-9c5b-420d78b88173 None admin] PATH_INFO: `/v3/users/cc50f7ddb6bc4a43be77cd9e32064022` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:10:03.680504 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63265708-3e82-439d-9c5b-420d78b88173 None admin] RBAC: Authorizing `identity:get_user(user_id=cc50f7ddb6bc4a43be77cd9e32064022)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:10:03.682425 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63265708-3e82-439d-9c5b-420d78b88173 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:10:03.684320 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 603/1197] 10.222.0.18 () {56 vars in 1177 bytes} [Wed Aug 24 17:10:03 2022] GET /identity/v3/users/cc50f7ddb6bc4a43be77cd9e32064022 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:10:03.690133 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253f7ab0-8feb-4b2f-9f5a-48e67f72c350 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:10:03.788304 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253f7ab0-8feb-4b2f-9f5a-48e67f72c350 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:10:03.820993 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30233e71-333e-4dc3-97c8-8e1516393887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:10:03.821588 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30233e71-333e-4dc3-97c8-8e1516393887 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:10:03.821869 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30233e71-333e-4dc3-97c8-8e1516393887 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:10:03.822143 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30233e71-333e-4dc3-97c8-8e1516393887 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:10:03.839747 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:10:03.888911 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30233e71-333e-4dc3-97c8-8e1516393887 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:10:03.890354 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30233e71-333e-4dc3-97c8-8e1516393887 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:10:03.985561 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 595/1198] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 24 17:10:03 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 17:10:03.991535 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63265708-3e82-439d-9c5b-420d78b88173 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:10:04.087626 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63265708-3e82-439d-9c5b-420d78b88173 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:10:04.120232 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d1c324-10fe-4d5d-ae64-8caed429b18c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:10:04.120838 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d1c324-10fe-4d5d-ae64-8caed429b18c None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:10:04.121114 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d1c324-10fe-4d5d-ae64-8caed429b18c None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:10:04.121383 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d1c324-10fe-4d5d-ae64-8caed429b18c None admin] PATH_INFO: `/v3/projects/c3de589d8a8b48e79e99730bb74a496f` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:10:04.122480 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d1c324-10fe-4d5d-ae64-8caed429b18c None admin] RBAC: Authorizing `identity:get_project(project_id=c3de589d8a8b48e79e99730bb74a496f)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:10:04.123922 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d1c324-10fe-4d5d-ae64-8caed429b18c None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:10:04.125501 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 604/1199] 10.222.0.18 () {56 vars in 1186 bytes} [Wed Aug 24 17:10:03 2022] GET /identity/v3/projects/c3de589d8a8b48e79e99730bb74a496f => generated 361 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:10:04.131741 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30233e71-333e-4dc3-97c8-8e1516393887 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:10:04.179742 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:10:04.228733 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30233e71-333e-4dc3-97c8-8e1516393887 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:10:04.262657 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e633663-b478-4447-99dd-75472c2bbcdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:10:04.263282 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e633663-b478-4447-99dd-75472c2bbcdb None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:10:04.263564 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e633663-b478-4447-99dd-75472c2bbcdb None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:10:04.263836 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e633663-b478-4447-99dd-75472c2bbcdb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:10:04.264558 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e633663-b478-4447-99dd-75472c2bbcdb None admin] RBAC: Adding query filter params (role.id=c5346bc54abf445f9900edf94264c850, scope.project.id=c3de589d8a8b48e79e99730bb74a496f, user.id=cc50f7ddb6bc4a43be77cd9e32064022) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:10:04.264867 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e633663-b478-4447-99dd-75472c2bbcdb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:10:04.266282 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e633663-b478-4447-99dd-75472c2bbcdb None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:10:04.271940 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 596/1200] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 24 17:10:04 2022] GET /identity/v3/role_assignments?user.id=cc50f7ddb6bc4a43be77cd9e32064022&scope.project.id=c3de589d8a8b48e79e99730bb74a496f&role.id=c5346bc54abf445f9900edf94264c850 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:10:14.115682 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=76 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:10:14.269385 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=44 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:11:16.560138 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc68bb91-bd3a-4089-b146-4e2162e8f442 None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:11:16.561252 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc68bb91-bd3a-4089-b146-4e2162e8f442 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:11:16.561367 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc68bb91-bd3a-4089-b146-4e2162e8f442 None None] PATH_INFO: `/` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:11:16.562637 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 605/1201] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:11:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 17:11:16.568102 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d3463f-bbbb-494b-bfc6-e26374800440 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:11:16.568497 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d3463f-bbbb-494b-bfc6-e26374800440 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:11:16.568650 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d3463f-bbbb-494b-bfc6-e26374800440 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:11:16.586386 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-73d3463f-bbbb-494b-bfc6-e26374800440 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:11:16.587662 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-73d3463f-bbbb-494b-bfc6-e26374800440 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:11:16.652794 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 597/1202] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:11:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:11:16.923456 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc68bb91-bd3a-4089-b146-4e2162e8f442 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:11:17.022829 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc68bb91-bd3a-4089-b146-4e2162e8f442 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:11:17.055840 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ae1fa0-9e09-4d39-8685-da373ccc07e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:11:17.056456 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ae1fa0-9e09-4d39-8685-da373ccc07e5 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:11:17.056735 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ae1fa0-9e09-4d39-8685-da373ccc07e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:11:17.057005 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ae1fa0-9e09-4d39-8685-da373ccc07e5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:11:17.067093 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ae1fa0-9e09-4d39-8685-da373ccc07e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:11:17.068514 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ae1fa0-9e09-4d39-8685-da373ccc07e5 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:11:17.082249 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-08ae1fa0-9e09-4d39-8685-da373ccc07e5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:11:17.082249 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:11:17.082249 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:11:17.082249 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:11:17.082249 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:11:17.082249 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:11:17.082249 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:11:17.082249 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:11:17.082249 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:11:17.082249 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:11:17.082249 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:11:17.082249 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:11:17.082249 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:11:17.082249 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:11:17.082249 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:11:17.082249 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:11:17.083458 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:11:17.083458 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:11:17.083458 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:11:17.083458 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:11:17.083458 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:11:17.083458 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:11:17.083458 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:11:17.083458 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application with Lock( Aug 24 17:11:17.083458 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:11:17.083458 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:11:17.083458 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:11:17.083458 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:11:17.083458 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:11:17.083458 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:11:17.083458 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:11:17.083458 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:11:17.083458 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:11:17.084584 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:11:17.084584 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:11:17.084584 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:11:17.084584 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 606/1203] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 24 17:11:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:11:17.089481 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d3463f-bbbb-494b-bfc6-e26374800440 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:11:17.189176 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d3463f-bbbb-494b-bfc6-e26374800440 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:11:17.217761 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:11:17.223334 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c7d8a9-cc0b-4551-9ceb-fd78e5443e7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:11:17.227562 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c7d8a9-cc0b-4551-9ceb-fd78e5443e7f None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:11:17.227562 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c7d8a9-cc0b-4551-9ceb-fd78e5443e7f None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:11:17.227562 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c7d8a9-cc0b-4551-9ceb-fd78e5443e7f None admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:11:17.227562 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c7d8a9-cc0b-4551-9ceb-fd78e5443e7f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:11:17.227562 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c7d8a9-cc0b-4551-9ceb-fd78e5443e7f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:11:17.227562 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c7d8a9-cc0b-4551-9ceb-fd78e5443e7f None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:11:17.237531 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 598/1204] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 24 17:11:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:11:17.243524 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ae1fa0-9e09-4d39-8685-da373ccc07e5 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:11:17.296581 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:11:17.342603 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ae1fa0-9e09-4d39-8685-da373ccc07e5 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:11:17.373830 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7198a9a-02e5-4407-9c6f-fa649608abaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:11:17.374496 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7198a9a-02e5-4407-9c6f-fa649608abaa None admin] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:11:17.374775 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7198a9a-02e5-4407-9c6f-fa649608abaa None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:11:17.375101 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7198a9a-02e5-4407-9c6f-fa649608abaa None admin] PATH_INFO: `/v3/users` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:11:17.380981 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7198a9a-02e5-4407-9c6f-fa649608abaa None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:11:17.380981 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7198a9a-02e5-4407-9c6f-fa649608abaa None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:11:17.450593 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 607/1205] 10.222.0.18 () {60 vars in 1132 bytes} [Wed Aug 24 17:11:17 2022] POST /identity/v3/users => generated 251 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:11:19.398992 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1cdafd-66bb-416a-a990-a72e012e3d03 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:11:19.399565 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1cdafd-66bb-416a-a990-a72e012e3d03 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:11:19.399565 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1cdafd-66bb-416a-a990-a72e012e3d03 None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:11:19.400897 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 599/1206] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:11:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:11:19.406743 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e40981e-f97d-4fdc-8907-3949ecdcded4 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:11:19.407149 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e40981e-f97d-4fdc-8907-3949ecdcded4 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:11:19.414651 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e40981e-f97d-4fdc-8907-3949ecdcded4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:11:19.425483 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-7e40981e-f97d-4fdc-8907-3949ecdcded4 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:11:19.426646 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-7e40981e-f97d-4fdc-8907-3949ecdcded4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:11:19.491900 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 608/1207] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:11:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:11:19.761583 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1cdafd-66bb-416a-a990-a72e012e3d03 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:11:19.862962 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1cdafd-66bb-416a-a990-a72e012e3d03 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:11:19.897411 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2689d9-f02a-4735-b821-cd26a4f3e5ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:11:19.898071 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2689d9-f02a-4735-b821-cd26a4f3e5ca None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:11:19.898310 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2689d9-f02a-4735-b821-cd26a4f3e5ca None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:11:19.898584 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2689d9-f02a-4735-b821-cd26a4f3e5ca None admin] PATH_INFO: `/v3/roles/service` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:11:19.908932 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2689d9-f02a-4735-b821-cd26a4f3e5ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:11:19.913604 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-9b2689d9-f02a-4735-b821-cd26a4f3e5ca None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:11:19.913604 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:11:19.913604 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:11:19.913604 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:11:19.913604 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:11:19.913604 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:11:19.913604 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:11:19.913604 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:11:19.913604 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:11:19.913604 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:11:19.913604 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:11:19.913604 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:11:19.913604 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:11:19.913604 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:11:19.913604 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:11:19.913604 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:11:19.914589 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:11:19.914589 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:11:19.914589 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:11:19.914589 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:11:19.914589 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:11:19.914589 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:11:19.914589 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:11:19.914589 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:11:19.914589 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:11:19.914589 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:11:19.914589 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:11:19.914589 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 24 17:11:19.914589 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:11:19.914589 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:11:19.914589 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:11:19.914589 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:11:19.915549 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:11:19.915549 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:11:19.915549 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:11:19.915549 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:11:19.915549 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:11:19.915549 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:11:19.915549 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:11:19.915549 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:11:19.915549 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:11:19.915549 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:11:19.915549 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:11:19.915549 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:11:19.915549 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 600/1208] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Aug 24 17:11:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:11:19.920946 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e40981e-f97d-4fdc-8907-3949ecdcded4 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:11:19.959007 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:11:20.015539 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e40981e-f97d-4fdc-8907-3949ecdcded4 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:11:20.046778 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d10ef18-18b8-4ef1-8eb5-2afc2b37e300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:11:20.047393 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d10ef18-18b8-4ef1-8eb5-2afc2b37e300 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:11:20.047669 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d10ef18-18b8-4ef1-8eb5-2afc2b37e300 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:11:20.047938 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d10ef18-18b8-4ef1-8eb5-2afc2b37e300 None admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:11:20.048614 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d10ef18-18b8-4ef1-8eb5-2afc2b37e300 None admin] RBAC: Adding query filter params (name=service) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:11:20.049174 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d10ef18-18b8-4ef1-8eb5-2afc2b37e300 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:11:20.050867 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d10ef18-18b8-4ef1-8eb5-2afc2b37e300 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:11:20.058978 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 609/1209] 10.222.0.18 () {56 vars in 1103 bytes} [Wed Aug 24 17:11:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:11:20.065334 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2689d9-f02a-4735-b821-cd26a4f3e5ca None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:11:20.114887 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:11:20.163698 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2689d9-f02a-4735-b821-cd26a4f3e5ca None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:11:20.200381 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad51a72-7153-45a4-9035-8021a2249a55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:11:20.201720 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad51a72-7153-45a4-9035-8021a2249a55 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:11:20.202070 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad51a72-7153-45a4-9035-8021a2249a55 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:11:20.202536 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad51a72-7153-45a4-9035-8021a2249a55 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:11:20.213535 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad51a72-7153-45a4-9035-8021a2249a55 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:11:20.214999 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad51a72-7153-45a4-9035-8021a2249a55 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:11:20.228607 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-cad51a72-7153-45a4-9035-8021a2249a55 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:11:20.228607 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:11:20.228607 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:11:20.228607 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:11:20.228607 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:11:20.228607 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:11:20.228607 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:11:20.228607 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:11:20.228607 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:11:20.228607 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:11:20.228607 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:11:20.228607 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:11:20.228607 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:11:20.228607 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:11:20.228607 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:11:20.228607 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:11:20.229723 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:11:20.229723 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:11:20.229723 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:11:20.229723 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:11:20.229723 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:11:20.229723 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:11:20.229723 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:11:20.229723 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 24 17:11:20.229723 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:11:20.229723 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:11:20.229723 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:11:20.229723 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:11:20.229723 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:11:20.229723 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:11:20.229723 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:11:20.229723 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:11:20.229723 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:11:20.231128 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:11:20.231128 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:11:20.231128 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:11:20.231128 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 601/1210] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 24 17:11:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:11:20.236129 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d10ef18-18b8-4ef1-8eb5-2afc2b37e300 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:11:20.331384 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:11:20.334359 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d10ef18-18b8-4ef1-8eb5-2afc2b37e300 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:11:20.367461 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0311e3-64f2-4451-a438-b9d583a95d99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:11:20.368047 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0311e3-64f2-4451-a438-b9d583a95d99 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:11:20.368324 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0311e3-64f2-4451-a438-b9d583a95d99 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:11:20.368592 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0311e3-64f2-4451-a438-b9d583a95d99 None admin] PATH_INFO: `/v3/domains` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:11:20.369220 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0311e3-64f2-4451-a438-b9d583a95d99 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:11:20.369519 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0311e3-64f2-4451-a438-b9d583a95d99 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:11:20.370857 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0311e3-64f2-4451-a438-b9d583a95d99 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:11:20.383530 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 610/1211] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 24 17:11:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:11:20.389358 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad51a72-7153-45a4-9035-8021a2249a55 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:11:20.495353 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad51a72-7153-45a4-9035-8021a2249a55 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:11:20.496917 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:11:20.528825 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d927cf66-a439-4392-9aec-07310e85a9f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:11:20.529444 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d927cf66-a439-4392-9aec-07310e85a9f1 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:11:20.529716 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d927cf66-a439-4392-9aec-07310e85a9f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:11:20.529979 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d927cf66-a439-4392-9aec-07310e85a9f1 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:11:20.537053 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d927cf66-a439-4392-9aec-07310e85a9f1 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:11:20.538468 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d927cf66-a439-4392-9aec-07310e85a9f1 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:11:20.552251 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-d927cf66-a439-4392-9aec-07310e85a9f1 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 17:11:20.552251 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:11:20.552251 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:11:20.552251 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:11:20.552251 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:11:20.552251 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:11:20.552251 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:11:20.552251 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:11:20.552251 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:11:20.552251 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:11:20.552251 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:11:20.552251 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:11:20.552251 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:11:20.552251 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:11:20.552251 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:11:20.552251 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:11:20.553354 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:11:20.553354 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:11:20.553354 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:11:20.553354 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:11:20.553354 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:11:20.553354 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:11:20.553354 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:11:20.553354 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:11:20.553354 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:11:20.553354 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:11:20.553354 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:11:20.553354 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 24 17:11:20.553354 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:11:20.553354 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:11:20.553354 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:11:20.553354 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:11:20.553354 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:11:20.553354 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:11:20.554350 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:11:20.554350 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:11:20.554350 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:11:20.554350 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:11:20.554350 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:11:20.554350 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:11:20.554350 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:11:20.554350 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:11:20.554350 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 17:11:20.554350 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:11:20.554350 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 602/1212] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 24 17:11:20 2022] GET /identity/v3/users/placement => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:11:20.557085 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0311e3-64f2-4451-a438-b9d583a95d99 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:11:20.655568 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0311e3-64f2-4451-a438-b9d583a95d99 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:11:20.688558 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d5d0d1-8316-4e97-937f-88c41093315b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:11:20.689155 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d5d0d1-8316-4e97-937f-88c41093315b None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:11:20.689436 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d5d0d1-8316-4e97-937f-88c41093315b None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:11:20.689723 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d5d0d1-8316-4e97-937f-88c41093315b None admin] PATH_INFO: `/v3/users` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:11:20.690539 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d5d0d1-8316-4e97-937f-88c41093315b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:11:20.690841 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d5d0d1-8316-4e97-937f-88c41093315b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:11:20.692277 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d5d0d1-8316-4e97-937f-88c41093315b None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:11:20.709037 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 611/1213] 10.222.0.18 () {56 vars in 1143 bytes} [Wed Aug 24 17:11:20 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:11:20.714852 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d927cf66-a439-4392-9aec-07310e85a9f1 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:11:20.812963 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d927cf66-a439-4392-9aec-07310e85a9f1 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:11:20.846604 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00036c87-80f9-4458-84a5-7be22c7cf081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:11:20.847232 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00036c87-80f9-4458-84a5-7be22c7cf081 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:11:20.847518 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00036c87-80f9-4458-84a5-7be22c7cf081 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:11:20.847792 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00036c87-80f9-4458-84a5-7be22c7cf081 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:11:20.878468 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:11:20.914527 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00036c87-80f9-4458-84a5-7be22c7cf081 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:11:20.915924 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00036c87-80f9-4458-84a5-7be22c7cf081 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:11:21.015268 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 603/1214] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 24 17:11:20 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 17:11:21.021195 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d5d0d1-8316-4e97-937f-88c41093315b None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:11:21.040424 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:11:21.121113 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d5d0d1-8316-4e97-937f-88c41093315b None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:11:21.155027 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09b7a2d-a5a5-4ad2-b313-13d5eb23feeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:11:21.155656 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09b7a2d-a5a5-4ad2-b313-13d5eb23feeb None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:11:21.155937 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09b7a2d-a5a5-4ad2-b313-13d5eb23feeb None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:11:21.156208 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09b7a2d-a5a5-4ad2-b313-13d5eb23feeb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:11:21.165944 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09b7a2d-a5a5-4ad2-b313-13d5eb23feeb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:11:21.167358 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09b7a2d-a5a5-4ad2-b313-13d5eb23feeb None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:11:21.185252 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-c09b7a2d-a5a5-4ad2-b313-13d5eb23feeb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:11:21.185252 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:11:21.185252 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:11:21.185252 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:11:21.185252 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:11:21.185252 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:11:21.185252 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:11:21.185252 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:11:21.185252 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:11:21.185252 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:11:21.185252 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:11:21.185252 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:11:21.185252 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:11:21.185252 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:11:21.185252 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:11:21.185252 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:11:21.186359 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:11:21.186359 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:11:21.186359 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:11:21.186359 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:11:21.186359 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:11:21.186359 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:11:21.186359 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:11:21.186359 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application with Lock( Aug 24 17:11:21.186359 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:11:21.186359 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:11:21.186359 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:11:21.186359 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:11:21.186359 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:11:21.186359 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:11:21.186359 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:11:21.186359 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:11:21.186359 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:11:21.187314 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:11:21.187314 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:11:21.187314 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:11:21.187314 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 612/1215] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 24 17:11:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:11:21.192149 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00036c87-80f9-4458-84a5-7be22c7cf081 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:11:21.253762 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:11:21.291521 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00036c87-80f9-4458-84a5-7be22c7cf081 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:11:21.324085 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbf0446-effc-4a84-92a7-db27b4c9a7c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:11:21.324700 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbf0446-effc-4a84-92a7-db27b4c9a7c1 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:11:21.324943 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbf0446-effc-4a84-92a7-db27b4c9a7c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:11:21.325222 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbf0446-effc-4a84-92a7-db27b4c9a7c1 None admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:11:21.325858 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbf0446-effc-4a84-92a7-db27b4c9a7c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:11:21.326688 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbf0446-effc-4a84-92a7-db27b4c9a7c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:11:21.328099 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbf0446-effc-4a84-92a7-db27b4c9a7c1 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:11:21.339258 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 604/1216] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 24 17:11:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:11:21.345595 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09b7a2d-a5a5-4ad2-b313-13d5eb23feeb None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:11:21.413168 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:11:21.442446 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09b7a2d-a5a5-4ad2-b313-13d5eb23feeb None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:11:21.475143 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771c3060-99b9-4e17-a320-dbc660f32b8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:11:21.475754 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771c3060-99b9-4e17-a320-dbc660f32b8d None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:11:21.476028 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771c3060-99b9-4e17-a320-dbc660f32b8d None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:11:21.476292 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771c3060-99b9-4e17-a320-dbc660f32b8d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:11:21.482572 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771c3060-99b9-4e17-a320-dbc660f32b8d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:11:21.484104 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771c3060-99b9-4e17-a320-dbc660f32b8d None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:11:21.494702 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-771c3060-99b9-4e17-a320-dbc660f32b8d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:11:21.494702 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:11:21.494702 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:11:21.494702 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:11:21.494702 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:11:21.494702 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:11:21.494702 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:11:21.494702 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:11:21.494702 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:11:21.494702 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:11:21.494702 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:11:21.494702 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:11:21.494702 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:11:21.494702 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:11:21.494702 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:11:21.495871 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:11:21.495871 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:11:21.495871 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:11:21.495871 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:11:21.495871 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:11:21.495871 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:11:21.495871 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:11:21.495871 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:11:21.495871 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application with Lock( Aug 24 17:11:21.495871 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:11:21.495871 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:11:21.495871 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:11:21.495871 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:11:21.495871 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:11:21.495871 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:11:21.495871 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:11:21.495871 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:11:21.497052 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:11:21.497052 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:11:21.497052 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:11:21.497052 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:11:21.497052 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:11:21.497052 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:11:21.497052 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:11:21.497052 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:11:21.497052 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 613/1217] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 24 17:11:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:11:21.502391 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbf0446-effc-4a84-92a7-db27b4c9a7c1 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:11:21.603478 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbf0446-effc-4a84-92a7-db27b4c9a7c1 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:11:21.625882 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:11:21.637578 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb0106c-3e7e-4c63-bd18-e0d4a16e4752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:11:21.638253 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb0106c-3e7e-4c63-bd18-e0d4a16e4752 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:11:21.638614 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb0106c-3e7e-4c63-bd18-e0d4a16e4752 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:11:21.639416 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb0106c-3e7e-4c63-bd18-e0d4a16e4752 None admin] PATH_INFO: `/v3/projects` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:11:21.640099 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb0106c-3e7e-4c63-bd18-e0d4a16e4752 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:11:21.640415 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb0106c-3e7e-4c63-bd18-e0d4a16e4752 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:11:21.641825 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb0106c-3e7e-4c63-bd18-e0d4a16e4752 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:11:21.654638 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 605/1218] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Aug 24 17:11:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:11:21.661435 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771c3060-99b9-4e17-a320-dbc660f32b8d None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:11:21.759901 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771c3060-99b9-4e17-a320-dbc660f32b8d None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:11:21.786808 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:11:21.792382 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7d032a-3ddf-4ffe-8865-4636ac44073e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:11:21.792971 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7d032a-3ddf-4ffe-8865-4636ac44073e None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:11:21.793251 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7d032a-3ddf-4ffe-8865-4636ac44073e None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:11:21.793521 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7d032a-3ddf-4ffe-8865-4636ac44073e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:11:21.794228 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7d032a-3ddf-4ffe-8865-4636ac44073e None admin] RBAC: Adding query filter params (role.id=e7db2a847f56438bac13d0ff21c96b73, scope.project.id=46ab158bc3644c888557781b0630b0b8, user.id=03fc77c6107e430f90f7b6f7cfd4ec02) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:11:21.794530 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7d032a-3ddf-4ffe-8865-4636ac44073e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:11:21.796493 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7d032a-3ddf-4ffe-8865-4636ac44073e None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:11:21.801727 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 614/1219] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 24 17:11:21 2022] GET /identity/v3/role_assignments?user.id=03fc77c6107e430f90f7b6f7cfd4ec02&scope.project.id=46ab158bc3644c888557781b0630b0b8&role.id=e7db2a847f56438bac13d0ff21c96b73 => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:11:23.676657 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e5f409-6e27-4ee3-8722-17834f1c9289 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:11:23.677208 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e5f409-6e27-4ee3-8722-17834f1c9289 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:11:23.677208 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e5f409-6e27-4ee3-8722-17834f1c9289 None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:11:23.678552 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 606/1220] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:11:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:11:23.683606 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17ed777-cf6c-466c-9dfb-84553a6cf1ae None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:11:23.684040 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17ed777-cf6c-466c-9dfb-84553a6cf1ae None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:11:23.684218 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17ed777-cf6c-466c-9dfb-84553a6cf1ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:11:23.701816 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-f17ed777-cf6c-466c-9dfb-84553a6cf1ae None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:11:23.703002 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-f17ed777-cf6c-466c-9dfb-84553a6cf1ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:11:23.769965 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 615/1221] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:11:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:11:24.037176 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e5f409-6e27-4ee3-8722-17834f1c9289 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:11:24.142995 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e5f409-6e27-4ee3-8722-17834f1c9289 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:11:24.177108 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d67a66-e245-44a7-afcb-65977816d6bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:11:24.177731 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d67a66-e245-44a7-afcb-65977816d6bb None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:11:24.178011 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d67a66-e245-44a7-afcb-65977816d6bb None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:11:24.178284 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d67a66-e245-44a7-afcb-65977816d6bb None admin] PATH_INFO: `/v3/roles/service` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:11:24.188309 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d67a66-e245-44a7-afcb-65977816d6bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:11:24.192960 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-78d67a66-e245-44a7-afcb-65977816d6bb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:11:24.192960 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:11:24.192960 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:11:24.192960 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:11:24.192960 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:11:24.192960 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:11:24.192960 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:11:24.192960 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:11:24.192960 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:11:24.192960 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:11:24.192960 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:11:24.192960 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:11:24.192960 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:11:24.192960 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:11:24.192960 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:11:24.192960 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:11:24.194056 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:11:24.194056 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:11:24.194056 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:11:24.194056 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:11:24.194056 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:11:24.194056 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:11:24.194056 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:11:24.194056 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:11:24.194056 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:11:24.194056 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:11:24.194056 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:11:24.194056 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 24 17:11:24.194056 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:11:24.194056 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:11:24.194056 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:11:24.194056 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:11:24.194966 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:11:24.194966 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:11:24.194966 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:11:24.194966 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:11:24.194966 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:11:24.194966 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:11:24.194966 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:11:24.194966 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:11:24.194966 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:11:24.194966 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:11:24.194966 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:11:24.194966 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:11:24.194966 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 607/1222] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Aug 24 17:11:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:11:24.200308 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17ed777-cf6c-466c-9dfb-84553a6cf1ae None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:11:24.306185 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17ed777-cf6c-466c-9dfb-84553a6cf1ae None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:11:24.328909 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:11:24.339102 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3f4b33-6342-4bb3-b805-4823050a25dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:11:24.339704 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3f4b33-6342-4bb3-b805-4823050a25dd None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:11:24.339984 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3f4b33-6342-4bb3-b805-4823050a25dd None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:11:24.340259 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3f4b33-6342-4bb3-b805-4823050a25dd None admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:11:24.340914 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3f4b33-6342-4bb3-b805-4823050a25dd None admin] RBAC: Adding query filter params (name=service) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:11:24.341217 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3f4b33-6342-4bb3-b805-4823050a25dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:11:24.343204 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3f4b33-6342-4bb3-b805-4823050a25dd None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:11:24.350668 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 616/1223] 10.222.0.18 () {56 vars in 1103 bytes} [Wed Aug 24 17:11:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:11:24.356661 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d67a66-e245-44a7-afcb-65977816d6bb None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:11:24.405197 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:11:24.462375 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d67a66-e245-44a7-afcb-65977816d6bb None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:11:24.495159 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee0f240-487f-427a-9084-162787118268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:11:24.495775 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee0f240-487f-427a-9084-162787118268 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:11:24.496050 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee0f240-487f-427a-9084-162787118268 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:11:24.496315 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee0f240-487f-427a-9084-162787118268 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:11:24.507129 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee0f240-487f-427a-9084-162787118268 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:11:24.508559 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee0f240-487f-427a-9084-162787118268 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:11:24.523527 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-0ee0f240-487f-427a-9084-162787118268 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:11:24.523527 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:11:24.523527 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:11:24.523527 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:11:24.523527 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:11:24.523527 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:11:24.523527 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:11:24.523527 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:11:24.523527 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:11:24.523527 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:11:24.523527 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:11:24.523527 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:11:24.523527 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:11:24.523527 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:11:24.523527 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:11:24.523527 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:11:24.529685 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:11:24.529685 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:11:24.529685 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:11:24.529685 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:11:24.529685 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:11:24.529685 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:11:24.529685 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:11:24.529685 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 24 17:11:24.529685 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:11:24.529685 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:11:24.529685 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:11:24.529685 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:11:24.529685 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:11:24.529685 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:11:24.529685 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:11:24.529685 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:11:24.529685 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:11:24.530882 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:11:24.530882 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:11:24.530882 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:11:24.530882 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 608/1224] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 24 17:11:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:11:24.531180 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3f4b33-6342-4bb3-b805-4823050a25dd None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:11:24.630857 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3f4b33-6342-4bb3-b805-4823050a25dd None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:11:24.663494 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67dc8e6-876b-4a97-b803-eb73fd6da4c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:11:24.664079 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67dc8e6-876b-4a97-b803-eb73fd6da4c0 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:11:24.664353 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67dc8e6-876b-4a97-b803-eb73fd6da4c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:11:24.664619 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67dc8e6-876b-4a97-b803-eb73fd6da4c0 None admin] PATH_INFO: `/v3/domains` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:11:24.665244 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67dc8e6-876b-4a97-b803-eb73fd6da4c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:11:24.665541 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67dc8e6-876b-4a97-b803-eb73fd6da4c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:11:24.666996 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67dc8e6-876b-4a97-b803-eb73fd6da4c0 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:11:24.678576 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 617/1225] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 24 17:11:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:11:24.684155 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee0f240-487f-427a-9084-162787118268 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:11:24.777937 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:11:24.784701 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee0f240-487f-427a-9084-162787118268 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:11:24.817936 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be034daf-4c98-4b72-a8e5-362566df80f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:11:24.818557 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be034daf-4c98-4b72-a8e5-362566df80f3 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:11:24.818845 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be034daf-4c98-4b72-a8e5-362566df80f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:11:24.819150 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be034daf-4c98-4b72-a8e5-362566df80f3 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:11:24.826289 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be034daf-4c98-4b72-a8e5-362566df80f3 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:11:24.827742 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be034daf-4c98-4b72-a8e5-362566df80f3 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:11:24.838388 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-be034daf-4c98-4b72-a8e5-362566df80f3 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 17:11:24.838388 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:11:24.838388 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:11:24.838388 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:11:24.838388 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:11:24.838388 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:11:24.838388 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:11:24.838388 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:11:24.838388 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:11:24.838388 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:11:24.838388 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:11:24.838388 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:11:24.838388 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:11:24.838388 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:11:24.838388 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:11:24.838388 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:11:24.839392 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:11:24.839392 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:11:24.839392 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:11:24.839392 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:11:24.839392 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:11:24.839392 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:11:24.839392 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:11:24.839392 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:11:24.839392 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:11:24.839392 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:11:24.839392 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:11:24.839392 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 24 17:11:24.839392 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:11:24.839392 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:11:24.839392 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:11:24.839392 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:11:24.839392 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:11:24.839392 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:11:24.840351 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:11:24.840351 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:11:24.840351 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:11:24.840351 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:11:24.840351 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:11:24.840351 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:11:24.840351 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:11:24.840351 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:11:24.840351 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 17:11:24.840351 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:11:24.840351 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 609/1226] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 24 17:11:24 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:11:24.845727 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67dc8e6-876b-4a97-b803-eb73fd6da4c0 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:11:24.891917 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:11:24.942602 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67dc8e6-876b-4a97-b803-eb73fd6da4c0 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:11:24.975329 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1bb8a6-0721-4846-9c4b-4f9d8bc88764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:11:24.975927 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1bb8a6-0721-4846-9c4b-4f9d8bc88764 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:11:24.976202 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1bb8a6-0721-4846-9c4b-4f9d8bc88764 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:11:24.976469 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1bb8a6-0721-4846-9c4b-4f9d8bc88764 None admin] PATH_INFO: `/v3/users` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:11:24.977210 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1bb8a6-0721-4846-9c4b-4f9d8bc88764 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:11:24.977513 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1bb8a6-0721-4846-9c4b-4f9d8bc88764 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:11:24.979090 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1bb8a6-0721-4846-9c4b-4f9d8bc88764 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:11:24.996402 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 618/1227] 10.222.0.18 () {56 vars in 1143 bytes} [Wed Aug 24 17:11:24 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:11:25.003183 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be034daf-4c98-4b72-a8e5-362566df80f3 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:11:25.101663 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be034daf-4c98-4b72-a8e5-362566df80f3 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:11:25.134908 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80694ee3-a924-4fd7-b56f-37631f5c9dc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:11:25.135556 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80694ee3-a924-4fd7-b56f-37631f5c9dc5 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:11:25.135837 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80694ee3-a924-4fd7-b56f-37631f5c9dc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:11:25.136109 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80694ee3-a924-4fd7-b56f-37631f5c9dc5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:11:25.147589 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80694ee3-a924-4fd7-b56f-37631f5c9dc5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:11:25.148923 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80694ee3-a924-4fd7-b56f-37631f5c9dc5 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:11:25.163194 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-80694ee3-a924-4fd7-b56f-37631f5c9dc5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:11:25.163194 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:11:25.163194 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:11:25.163194 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:11:25.163194 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:11:25.163194 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:11:25.163194 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:11:25.163194 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:11:25.163194 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:11:25.163194 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:11:25.163194 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:11:25.163194 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:11:25.163194 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:11:25.163194 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:11:25.163194 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:11:25.163194 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:11:25.165841 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:11:25.165841 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:11:25.165841 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:11:25.165841 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:11:25.165841 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:11:25.165841 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:11:25.165841 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:11:25.165841 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 24 17:11:25.165841 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:11:25.165841 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:11:25.165841 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:11:25.165841 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:11:25.165841 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:11:25.165841 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:11:25.165841 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:11:25.165841 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:11:25.165841 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:11:25.166980 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:11:25.166980 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:11:25.166980 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:11:25.166980 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 610/1228] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 24 17:11:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:11:25.171110 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1bb8a6-0721-4846-9c4b-4f9d8bc88764 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:11:25.279922 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1bb8a6-0721-4846-9c4b-4f9d8bc88764 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:11:25.282198 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:11:25.312916 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c1eb8a-06a5-48f0-8d48-4d79263197d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:11:25.313516 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c1eb8a-06a5-48f0-8d48-4d79263197d2 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:11:25.313792 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c1eb8a-06a5-48f0-8d48-4d79263197d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:11:25.314060 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c1eb8a-06a5-48f0-8d48-4d79263197d2 None admin] PATH_INFO: `/v3/domains` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:11:25.314691 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c1eb8a-06a5-48f0-8d48-4d79263197d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:11:25.315023 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c1eb8a-06a5-48f0-8d48-4d79263197d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:11:25.316527 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c1eb8a-06a5-48f0-8d48-4d79263197d2 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:11:25.327264 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 619/1229] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 24 17:11:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:11:25.333574 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80694ee3-a924-4fd7-b56f-37631f5c9dc5 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:11:25.342949 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:11:25.432139 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80694ee3-a924-4fd7-b56f-37631f5c9dc5 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:11:25.465197 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc004ea9-85b4-4e79-8b57-534f62a4f53b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:11:25.465813 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc004ea9-85b4-4e79-8b57-534f62a4f53b None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:11:25.466089 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc004ea9-85b4-4e79-8b57-534f62a4f53b None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:11:25.466359 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc004ea9-85b4-4e79-8b57-534f62a4f53b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:11:25.473339 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc004ea9-85b4-4e79-8b57-534f62a4f53b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:11:25.474752 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc004ea9-85b4-4e79-8b57-534f62a4f53b None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:11:25.485790 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-fc004ea9-85b4-4e79-8b57-534f62a4f53b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:11:25.485790 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:11:25.485790 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:11:25.485790 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:11:25.485790 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:11:25.485790 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:11:25.485790 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:11:25.485790 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:11:25.485790 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:11:25.485790 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:11:25.485790 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:11:25.485790 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:11:25.485790 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:11:25.485790 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:11:25.485790 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:11:25.486858 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:11:25.486858 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:11:25.486858 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:11:25.486858 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:11:25.486858 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:11:25.486858 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:11:25.486858 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:11:25.486858 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:11:25.486858 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 24 17:11:25.486858 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:11:25.486858 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:11:25.486858 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:11:25.486858 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:11:25.486858 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:11:25.486858 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:11:25.486858 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:11:25.486858 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:11:25.488142 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:11:25.488142 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:11:25.488142 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:11:25.488142 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:11:25.488142 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:11:25.488142 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:11:25.488142 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:11:25.488142 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:11:25.488142 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 611/1230] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 24 17:11:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:11:25.492966 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c1eb8a-06a5-48f0-8d48-4d79263197d2 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:11:25.597343 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c1eb8a-06a5-48f0-8d48-4d79263197d2 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:11:25.629767 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c86db2-176f-44fb-9a0f-ee0839bbfd96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:11:25.630367 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c86db2-176f-44fb-9a0f-ee0839bbfd96 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:11:25.630646 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c86db2-176f-44fb-9a0f-ee0839bbfd96 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:11:25.630937 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c86db2-176f-44fb-9a0f-ee0839bbfd96 None admin] PATH_INFO: `/v3/projects` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:11:25.631623 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c86db2-176f-44fb-9a0f-ee0839bbfd96 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:11:25.631927 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c86db2-176f-44fb-9a0f-ee0839bbfd96 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:11:25.633411 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c86db2-176f-44fb-9a0f-ee0839bbfd96 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:11:25.645519 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 620/1231] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Aug 24 17:11:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:11:25.651569 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc004ea9-85b4-4e79-8b57-534f62a4f53b None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:11:25.720607 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:11:25.754373 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc004ea9-85b4-4e79-8b57-534f62a4f53b None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:11:25.787043 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd901b0-6659-4559-9824-b08d47842fff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:11:25.787694 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd901b0-6659-4559-9824-b08d47842fff None admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:11:25.787976 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd901b0-6659-4559-9824-b08d47842fff None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:11:25.788251 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd901b0-6659-4559-9824-b08d47842fff None admin] PATH_INFO: `/v3/projects/46ab158bc3644c888557781b0630b0b8/users/03fc77c6107e430f90f7b6f7cfd4ec02/roles/e7db2a847f56438bac13d0ff21c96b73` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:11:25.801746 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd901b0-6659-4559-9824-b08d47842fff None admin] RBAC: Authorizing `identity:create_grant(project_id=46ab158bc3644c888557781b0630b0b8, user_id=03fc77c6107e430f90f7b6f7cfd4ec02, role_id=e7db2a847f56438bac13d0ff21c96b73)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:11:25.803208 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd901b0-6659-4559-9824-b08d47842fff None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:11:25.874214 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 612/1232] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Aug 24 17:11:25 2022] PUT /identity/v3/projects/46ab158bc3644c888557781b0630b0b8/users/03fc77c6107e430f90f7b6f7cfd4ec02/roles/e7db2a847f56438bac13d0ff21c96b73 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:11:27.786597 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91a57ea-ecd6-47de-94c3-db047808950c None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:11:27.787206 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91a57ea-ecd6-47de-94c3-db047808950c None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:11:27.787206 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91a57ea-ecd6-47de-94c3-db047808950c None None] PATH_INFO: `/` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:11:27.788515 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 621/1233] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:11:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:11:27.793770 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442b8bc9-1e08-4026-ba22-133c52a25f21 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:11:27.794147 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442b8bc9-1e08-4026-ba22-133c52a25f21 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:11:27.794324 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442b8bc9-1e08-4026-ba22-133c52a25f21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:11:27.812318 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-442b8bc9-1e08-4026-ba22-133c52a25f21 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:11:27.813437 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-442b8bc9-1e08-4026-ba22-133c52a25f21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:11:27.880855 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 613/1234] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:11:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:11:28.149311 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91a57ea-ecd6-47de-94c3-db047808950c None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:11:28.174131 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:11:28.246631 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91a57ea-ecd6-47de-94c3-db047808950c None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:11:28.285354 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b54bbbf-c724-4f85-ac78-79d886709c5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:11:28.285983 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b54bbbf-c724-4f85-ac78-79d886709c5e None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:11:28.286260 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b54bbbf-c724-4f85-ac78-79d886709c5e None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:11:28.286536 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b54bbbf-c724-4f85-ac78-79d886709c5e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:11:28.298689 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b54bbbf-c724-4f85-ac78-79d886709c5e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:11:28.303453 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-2b54bbbf-c724-4f85-ac78-79d886709c5e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:11:28.303453 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:11:28.303453 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:11:28.303453 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:11:28.303453 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:11:28.303453 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:11:28.303453 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:11:28.303453 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:11:28.303453 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:11:28.303453 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:11:28.303453 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:11:28.303453 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:11:28.303453 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:11:28.303453 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:11:28.303453 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:11:28.303453 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:11:28.304550 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:11:28.304550 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:11:28.304550 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:11:28.304550 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:11:28.304550 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:11:28.304550 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:11:28.304550 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:11:28.304550 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:11:28.304550 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:11:28.304550 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:11:28.304550 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:11:28.304550 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application with Lock( Aug 24 17:11:28.304550 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:11:28.304550 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:11:28.304550 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:11:28.304550 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:11:28.305799 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:11:28.305799 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:11:28.305799 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:11:28.305799 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:11:28.305799 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:11:28.305799 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:11:28.305799 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:11:28.305799 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:11:28.305799 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:11:28.305799 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:11:28.305799 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 17:11:28.305799 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:11:28.305799 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 622/1235] 10.222.0.18 () {56 vars in 1102 bytes} [Wed Aug 24 17:11:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:11:28.310688 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442b8bc9-1e08-4026-ba22-133c52a25f21 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:11:28.361022 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:11:28.412698 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442b8bc9-1e08-4026-ba22-133c52a25f21 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:11:28.446025 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841df204-18da-4041-a079-3e118c097daf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:11:28.446626 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841df204-18da-4041-a079-3e118c097daf None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:11:28.446925 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841df204-18da-4041-a079-3e118c097daf None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:11:28.447211 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841df204-18da-4041-a079-3e118c097daf None admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:11:28.447875 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841df204-18da-4041-a079-3e118c097daf None admin] RBAC: Adding query filter params (name=service) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:11:28.448164 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841df204-18da-4041-a079-3e118c097daf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:11:28.449614 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841df204-18da-4041-a079-3e118c097daf None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:11:28.457705 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 614/1236] 10.222.0.18 () {56 vars in 1103 bytes} [Wed Aug 24 17:11:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:11:28.463615 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b54bbbf-c724-4f85-ac78-79d886709c5e None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:11:28.547497 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:11:28.563453 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b54bbbf-c724-4f85-ac78-79d886709c5e None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:11:28.597141 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0c5a4e-25fb-4287-94a0-82c6a0a5ef51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:11:28.597860 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0c5a4e-25fb-4287-94a0-82c6a0a5ef51 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:11:28.598160 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0c5a4e-25fb-4287-94a0-82c6a0a5ef51 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:11:28.598428 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0c5a4e-25fb-4287-94a0-82c6a0a5ef51 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:11:28.608691 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0c5a4e-25fb-4287-94a0-82c6a0a5ef51 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:11:28.610123 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0c5a4e-25fb-4287-94a0-82c6a0a5ef51 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:11:28.623848 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-6a0c5a4e-25fb-4287-94a0-82c6a0a5ef51 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:11:28.623848 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:11:28.623848 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:11:28.623848 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:11:28.623848 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:11:28.623848 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:11:28.623848 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:11:28.623848 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:11:28.623848 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:11:28.623848 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:11:28.623848 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:11:28.623848 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:11:28.623848 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:11:28.623848 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:11:28.623848 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:11:28.623848 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:11:28.624908 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:11:28.624908 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:11:28.624908 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:11:28.624908 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:11:28.624908 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:11:28.624908 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:11:28.624908 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:11:28.624908 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application with Lock( Aug 24 17:11:28.624908 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:11:28.624908 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:11:28.624908 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:11:28.624908 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:11:28.624908 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:11:28.624908 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:11:28.624908 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:11:28.624908 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:11:28.624908 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:11:28.625884 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:11:28.625884 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:11:28.625884 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:11:28.625884 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 623/1237] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 24 17:11:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:11:28.631151 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841df204-18da-4041-a079-3e118c097daf None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:11:28.729809 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841df204-18da-4041-a079-3e118c097daf None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:11:28.738226 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:11:28.762679 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64aaedd9-89f5-4e99-8105-5b2153305299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:11:28.763308 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aaedd9-89f5-4e99-8105-5b2153305299 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:11:28.763594 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aaedd9-89f5-4e99-8105-5b2153305299 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:11:28.763866 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aaedd9-89f5-4e99-8105-5b2153305299 None admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:11:28.764498 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64aaedd9-89f5-4e99-8105-5b2153305299 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:11:28.765297 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64aaedd9-89f5-4e99-8105-5b2153305299 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:11:28.766760 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64aaedd9-89f5-4e99-8105-5b2153305299 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:11:28.778315 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 615/1238] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 24 17:11:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:11:28.784842 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0c5a4e-25fb-4287-94a0-82c6a0a5ef51 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:11:28.884500 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0c5a4e-25fb-4287-94a0-82c6a0a5ef51 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:11:28.923536 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c5f22d-6bb0-4894-8794-698939179266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:11:28.924201 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c5f22d-6bb0-4894-8794-698939179266 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:11:28.924479 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c5f22d-6bb0-4894-8794-698939179266 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:11:28.924750 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c5f22d-6bb0-4894-8794-698939179266 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:11:28.927866 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:11:28.933625 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c5f22d-6bb0-4894-8794-698939179266 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:11:28.935462 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c5f22d-6bb0-4894-8794-698939179266 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:11:28.946961 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-06c5f22d-6bb0-4894-8794-698939179266 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 17:11:28.946961 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:11:28.946961 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:11:28.946961 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:11:28.946961 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:11:28.946961 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:11:28.946961 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:11:28.946961 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:11:28.946961 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:11:28.946961 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:11:28.946961 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:11:28.946961 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:11:28.946961 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:11:28.946961 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:11:28.946961 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:11:28.946961 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:11:28.948963 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:11:28.948963 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:11:28.948963 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:11:28.948963 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:11:28.948963 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:11:28.948963 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:11:28.948963 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:11:28.948963 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:11:28.948963 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:11:28.948963 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:11:28.948963 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:11:28.948963 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application with Lock( Aug 24 17:11:28.948963 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:11:28.948963 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:11:28.948963 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:11:28.948963 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:11:28.948963 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:11:28.948963 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:11:28.950950 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:11:28.950950 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:11:28.950950 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:11:28.950950 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:11:28.950950 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:11:28.950950 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:11:28.950950 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:11:28.950950 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:11:28.950950 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 17:11:28.950950 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:11:28.952048 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 624/1239] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 24 17:11:28 2022] GET /identity/v3/users/placement => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:11:28.954799 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64aaedd9-89f5-4e99-8105-5b2153305299 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:11:29.054177 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64aaedd9-89f5-4e99-8105-5b2153305299 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:11:29.087811 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbef49b-13df-4175-98d4-a7780f4b549c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:11:29.088408 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbef49b-13df-4175-98d4-a7780f4b549c None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:11:29.088697 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbef49b-13df-4175-98d4-a7780f4b549c None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:11:29.088951 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbef49b-13df-4175-98d4-a7780f4b549c None admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:11:29.089664 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbef49b-13df-4175-98d4-a7780f4b549c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:11:29.089962 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbef49b-13df-4175-98d4-a7780f4b549c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:11:29.091418 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbef49b-13df-4175-98d4-a7780f4b549c None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:11:29.107831 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 616/1240] 10.222.0.18 () {56 vars in 1143 bytes} [Wed Aug 24 17:11:28 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:11:29.113791 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c5f22d-6bb0-4894-8794-698939179266 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:11:29.221562 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c5f22d-6bb0-4894-8794-698939179266 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:11:29.257007 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb41eac-201e-4739-93dc-6722c4babd17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:11:29.257609 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb41eac-201e-4739-93dc-6722c4babd17 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:11:29.257888 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb41eac-201e-4739-93dc-6722c4babd17 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:11:29.258160 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb41eac-201e-4739-93dc-6722c4babd17 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:11:29.320181 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:11:29.326643 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb41eac-201e-4739-93dc-6722c4babd17 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:11:29.328225 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb41eac-201e-4739-93dc-6722c4babd17 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:11:29.424635 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 625/1241] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 24 17:11:29 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 17:11:29.430628 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbef49b-13df-4175-98d4-a7780f4b549c None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:11:29.457642 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:11:29.527557 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbef49b-13df-4175-98d4-a7780f4b549c None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:11:29.564935 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52db87a1-f693-46d3-9ddf-832bbbffd80f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:11:29.567493 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52db87a1-f693-46d3-9ddf-832bbbffd80f None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:11:29.567493 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52db87a1-f693-46d3-9ddf-832bbbffd80f None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:11:29.567493 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52db87a1-f693-46d3-9ddf-832bbbffd80f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:11:29.576044 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52db87a1-f693-46d3-9ddf-832bbbffd80f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:11:29.577419 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52db87a1-f693-46d3-9ddf-832bbbffd80f None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:11:29.591479 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-52db87a1-f693-46d3-9ddf-832bbbffd80f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:11:29.591479 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:11:29.591479 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:11:29.591479 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:11:29.591479 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:11:29.591479 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:11:29.591479 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:11:29.591479 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:11:29.591479 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:11:29.591479 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:11:29.591479 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:11:29.591479 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:11:29.591479 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:11:29.591479 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:11:29.591479 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:11:29.591479 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:11:29.592650 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:11:29.592650 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:11:29.592650 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:11:29.592650 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:11:29.592650 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:11:29.592650 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:11:29.592650 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:11:29.592650 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 24 17:11:29.592650 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:11:29.592650 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:11:29.592650 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:11:29.592650 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:11:29.592650 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:11:29.592650 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:11:29.592650 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:11:29.592650 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:11:29.592650 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:11:29.594279 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:11:29.594279 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:11:29.594279 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:11:29.594279 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 617/1242] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 24 17:11:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:11:29.599021 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb41eac-201e-4739-93dc-6722c4babd17 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:11:29.689683 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:11:29.695330 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb41eac-201e-4739-93dc-6722c4babd17 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:11:29.728327 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae66187a-3115-4ae0-aa8b-a99bad5a1cc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:11:29.728930 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae66187a-3115-4ae0-aa8b-a99bad5a1cc6 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:11:29.729216 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae66187a-3115-4ae0-aa8b-a99bad5a1cc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:11:29.729488 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae66187a-3115-4ae0-aa8b-a99bad5a1cc6 None admin] PATH_INFO: `/v3/domains` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:11:29.730151 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae66187a-3115-4ae0-aa8b-a99bad5a1cc6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:11:29.730448 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae66187a-3115-4ae0-aa8b-a99bad5a1cc6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:11:29.731821 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae66187a-3115-4ae0-aa8b-a99bad5a1cc6 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:11:29.743281 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 626/1243] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 24 17:11:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:11:29.749267 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52db87a1-f693-46d3-9ddf-832bbbffd80f None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:11:29.828151 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:11:29.849496 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52db87a1-f693-46d3-9ddf-832bbbffd80f None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:11:29.883381 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d1dd9b-b5fb-4018-987d-c811c4c0d63d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:11:29.884033 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d1dd9b-b5fb-4018-987d-c811c4c0d63d None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:11:29.884314 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d1dd9b-b5fb-4018-987d-c811c4c0d63d None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:11:29.884589 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d1dd9b-b5fb-4018-987d-c811c4c0d63d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:11:29.891122 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d1dd9b-b5fb-4018-987d-c811c4c0d63d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:11:29.892545 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d1dd9b-b5fb-4018-987d-c811c4c0d63d None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:11:29.903559 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-d7d1dd9b-b5fb-4018-987d-c811c4c0d63d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:11:29.903559 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:11:29.903559 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:11:29.903559 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:11:29.903559 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:11:29.903559 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:11:29.903559 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:11:29.903559 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:11:29.903559 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:11:29.903559 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:11:29.903559 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:11:29.903559 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:11:29.903559 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:11:29.903559 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:11:29.903559 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:11:29.904624 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:11:29.904624 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:11:29.904624 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:11:29.904624 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:11:29.904624 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:11:29.904624 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:11:29.904624 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:11:29.904624 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:11:29.904624 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 24 17:11:29.904624 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:11:29.904624 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:11:29.904624 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:11:29.904624 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:11:29.904624 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:11:29.904624 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:11:29.904624 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:11:29.904624 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:11:29.906240 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:11:29.906240 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:11:29.906240 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:11:29.906240 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:11:29.906240 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:11:29.906240 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:11:29.906240 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:11:29.906240 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:11:29.906240 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 618/1244] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 24 17:11:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:11:29.911257 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae66187a-3115-4ae0-aa8b-a99bad5a1cc6 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:11:30.008834 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae66187a-3115-4ae0-aa8b-a99bad5a1cc6 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:11:30.042352 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9ec589-cc8b-437f-80fd-c6d6240d5dde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:11:30.042983 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9ec589-cc8b-437f-80fd-c6d6240d5dde None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:11:30.043269 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9ec589-cc8b-437f-80fd-c6d6240d5dde None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:11:30.043544 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9ec589-cc8b-437f-80fd-c6d6240d5dde None admin] PATH_INFO: `/v3/projects` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:11:30.044231 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9ec589-cc8b-437f-80fd-c6d6240d5dde None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:11:30.044537 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9ec589-cc8b-437f-80fd-c6d6240d5dde None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:11:30.045960 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9ec589-cc8b-437f-80fd-c6d6240d5dde None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:11:30.057852 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 627/1245] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Aug 24 17:11:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:11:30.064088 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d1dd9b-b5fb-4018-987d-c811c4c0d63d None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:11:30.165033 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d1dd9b-b5fb-4018-987d-c811c4c0d63d None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:11:30.197917 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e15e479-8c04-46c7-92ca-f25073dc99e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:11:30.198506 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e15e479-8c04-46c7-92ca-f25073dc99e9 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:11:30.198781 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e15e479-8c04-46c7-92ca-f25073dc99e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:11:30.199080 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e15e479-8c04-46c7-92ca-f25073dc99e9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:11:30.199792 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e15e479-8c04-46c7-92ca-f25073dc99e9 None admin] RBAC: Adding query filter params (role.id=e7db2a847f56438bac13d0ff21c96b73, scope.project.id=46ab158bc3644c888557781b0630b0b8, user.id=03fc77c6107e430f90f7b6f7cfd4ec02) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:11:30.200094 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e15e479-8c04-46c7-92ca-f25073dc99e9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:11:30.201577 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e15e479-8c04-46c7-92ca-f25073dc99e9 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:11:30.207431 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 619/1246] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 24 17:11:30 2022] GET /identity/v3/role_assignments?user.id=03fc77c6107e430f90f7b6f7cfd4ec02&scope.project.id=46ab158bc3644c888557781b0630b0b8&role.id=e7db2a847f56438bac13d0ff21c96b73 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:11:32.137580 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f841e3e-7396-43ff-a0bd-384dbf4e1b78 None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:11:32.138107 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f841e3e-7396-43ff-a0bd-384dbf4e1b78 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:11:32.138107 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f841e3e-7396-43ff-a0bd-384dbf4e1b78 None None] PATH_INFO: `/` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:11:32.139515 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 628/1247] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:11:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:11:32.144842 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426ad8f5-c905-4589-aacf-d09946c106de None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:11:32.145269 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426ad8f5-c905-4589-aacf-d09946c106de None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:11:32.145391 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426ad8f5-c905-4589-aacf-d09946c106de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:11:32.151842 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:11:32.164031 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-426ad8f5-c905-4589-aacf-d09946c106de None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:11:32.165211 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-426ad8f5-c905-4589-aacf-d09946c106de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:11:32.231392 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 620/1248] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:11:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:11:32.505775 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f841e3e-7396-43ff-a0bd-384dbf4e1b78 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:11:32.524258 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:11:32.607500 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f841e3e-7396-43ff-a0bd-384dbf4e1b78 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:11:32.641857 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4156c239-daa5-4c0c-93e9-5c874ad9a579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:11:32.642472 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4156c239-daa5-4c0c-93e9-5c874ad9a579 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:11:32.642746 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4156c239-daa5-4c0c-93e9-5c874ad9a579 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:11:32.643044 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4156c239-daa5-4c0c-93e9-5c874ad9a579 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:11:32.652665 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4156c239-daa5-4c0c-93e9-5c874ad9a579 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:11:32.657844 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-4156c239-daa5-4c0c-93e9-5c874ad9a579 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:11:32.657844 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:11:32.657844 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:11:32.657844 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:11:32.657844 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:11:32.657844 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:11:32.657844 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:11:32.657844 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:11:32.657844 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:11:32.657844 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:11:32.657844 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:11:32.657844 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:11:32.657844 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:11:32.657844 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:11:32.657844 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:11:32.657844 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:11:32.658959 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:11:32.658959 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:11:32.658959 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:11:32.658959 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:11:32.658959 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:11:32.658959 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:11:32.658959 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:11:32.658959 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:11:32.658959 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:11:32.658959 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:11:32.658959 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:11:32.658959 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application with Lock( Aug 24 17:11:32.658959 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:11:32.658959 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:11:32.658959 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:11:32.658959 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:11:32.660016 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:11:32.660016 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:11:32.660016 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:11:32.660016 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:11:32.660016 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:11:32.660016 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:11:32.660016 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:11:32.660016 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:11:32.660016 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:11:32.660016 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:11:32.660016 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:11:32.660016 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:11:32.660016 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 629/1249] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Aug 24 17:11:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:11:32.665177 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426ad8f5-c905-4589-aacf-d09946c106de None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:11:32.762795 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426ad8f5-c905-4589-aacf-d09946c106de None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:11:32.791456 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:11:32.796925 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6deec92-2673-4692-96aa-3267eefb7ddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:11:32.804903 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6deec92-2673-4692-96aa-3267eefb7ddd None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:11:32.804903 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6deec92-2673-4692-96aa-3267eefb7ddd None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:11:32.804903 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6deec92-2673-4692-96aa-3267eefb7ddd None admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:11:32.804903 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6deec92-2673-4692-96aa-3267eefb7ddd None admin] RBAC: Adding query filter params (name=admin) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:11:32.804903 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6deec92-2673-4692-96aa-3267eefb7ddd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:11:32.804903 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6deec92-2673-4692-96aa-3267eefb7ddd None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:11:32.808678 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 621/1250] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 24 17:11:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:11:32.815779 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4156c239-daa5-4c0c-93e9-5c874ad9a579 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:11:32.890916 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:11:32.913395 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4156c239-daa5-4c0c-93e9-5c874ad9a579 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:11:32.946040 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b06a6d-d36b-4884-993c-31155dab3514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:11:32.946664 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b06a6d-d36b-4884-993c-31155dab3514 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:11:32.946961 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b06a6d-d36b-4884-993c-31155dab3514 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:11:32.952731 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b06a6d-d36b-4884-993c-31155dab3514 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:11:32.957960 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b06a6d-d36b-4884-993c-31155dab3514 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:11:32.959420 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b06a6d-d36b-4884-993c-31155dab3514 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:11:32.973622 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-f4b06a6d-d36b-4884-993c-31155dab3514 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:11:32.973622 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:11:32.973622 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:11:32.973622 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:11:32.973622 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:11:32.973622 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:11:32.973622 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:11:32.973622 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:11:32.973622 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:11:32.973622 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:11:32.973622 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:11:32.973622 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:11:32.973622 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:11:32.973622 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:11:32.973622 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:11:32.973622 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:11:32.974662 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:11:32.974662 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:11:32.974662 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:11:32.974662 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:11:32.974662 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:11:32.974662 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:11:32.974662 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:11:32.974662 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application with Lock( Aug 24 17:11:32.974662 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:11:32.974662 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:11:32.974662 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:11:32.974662 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:11:32.974662 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:11:32.974662 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:11:32.974662 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:11:32.974662 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:11:32.974662 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:11:32.975851 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:11:32.975851 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:11:32.975851 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:11:32.975851 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 630/1251] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 24 17:11:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:11:32.980687 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6deec92-2673-4692-96aa-3267eefb7ddd None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:11:33.078766 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6deec92-2673-4692-96aa-3267eefb7ddd None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:11:33.111487 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50862012-5caf-4c09-a924-c8dd7a53ea0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:11:33.112079 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50862012-5caf-4c09-a924-c8dd7a53ea0a None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:11:33.112362 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50862012-5caf-4c09-a924-c8dd7a53ea0a None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:11:33.112633 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50862012-5caf-4c09-a924-c8dd7a53ea0a None admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:11:33.113261 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50862012-5caf-4c09-a924-c8dd7a53ea0a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:11:33.113562 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50862012-5caf-4c09-a924-c8dd7a53ea0a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:11:33.114928 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50862012-5caf-4c09-a924-c8dd7a53ea0a None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:11:33.126335 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 622/1252] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 24 17:11:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:11:33.132316 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b06a6d-d36b-4884-993c-31155dab3514 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:11:33.230660 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b06a6d-d36b-4884-993c-31155dab3514 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:11:33.257098 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:11:33.265114 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa97001-fc17-4d12-bead-50aa0e5cd77a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:11:33.265742 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa97001-fc17-4d12-bead-50aa0e5cd77a None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:11:33.266019 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa97001-fc17-4d12-bead-50aa0e5cd77a None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:11:33.266286 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa97001-fc17-4d12-bead-50aa0e5cd77a None admin] PATH_INFO: `/v3/users/placement` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:11:33.274214 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa97001-fc17-4d12-bead-50aa0e5cd77a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:11:33.275680 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa97001-fc17-4d12-bead-50aa0e5cd77a None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:11:33.287351 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-9aa97001-fc17-4d12-bead-50aa0e5cd77a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 17:11:33.287351 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:11:33.287351 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:11:33.287351 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:11:33.287351 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:11:33.287351 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:11:33.287351 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:11:33.287351 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:11:33.287351 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:11:33.287351 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:11:33.287351 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:11:33.287351 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:11:33.287351 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:11:33.287351 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:11:33.287351 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:11:33.287351 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:11:33.288520 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:11:33.288520 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:11:33.288520 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:11:33.288520 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:11:33.288520 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:11:33.288520 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:11:33.288520 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:11:33.288520 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:11:33.288520 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:11:33.288520 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:11:33.288520 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:11:33.288520 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application with Lock( Aug 24 17:11:33.288520 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:11:33.288520 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:11:33.288520 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:11:33.288520 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:11:33.288520 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:11:33.288520 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:11:33.289716 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:11:33.289716 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:11:33.289716 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:11:33.289716 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:11:33.289716 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:11:33.289716 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:11:33.289716 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:11:33.289716 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:11:33.289716 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 17:11:33.289716 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:11:33.292892 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 631/1253] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 24 17:11:33 2022] GET /identity/v3/users/placement => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:11:33.299683 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50862012-5caf-4c09-a924-c8dd7a53ea0a None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:11:33.355429 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:11:33.399487 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50862012-5caf-4c09-a924-c8dd7a53ea0a None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:11:33.432593 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ea35e3-cf19-4296-82f7-88d414c3c7a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:11:33.433197 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ea35e3-cf19-4296-82f7-88d414c3c7a3 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:11:33.433479 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ea35e3-cf19-4296-82f7-88d414c3c7a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:11:33.433750 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ea35e3-cf19-4296-82f7-88d414c3c7a3 None admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:11:33.434469 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ea35e3-cf19-4296-82f7-88d414c3c7a3 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:11:33.434771 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ea35e3-cf19-4296-82f7-88d414c3c7a3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:11:33.436217 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ea35e3-cf19-4296-82f7-88d414c3c7a3 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:11:33.453831 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 623/1254] 10.222.0.18 () {56 vars in 1143 bytes} [Wed Aug 24 17:11:33 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:11:33.459860 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa97001-fc17-4d12-bead-50aa0e5cd77a None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:11:33.559201 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa97001-fc17-4d12-bead-50aa0e5cd77a None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:11:33.593455 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edf98a1-b7d7-42c1-9426-2d504fe103df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:11:33.594472 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edf98a1-b7d7-42c1-9426-2d504fe103df None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:11:33.594753 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edf98a1-b7d7-42c1-9426-2d504fe103df None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:11:33.595050 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edf98a1-b7d7-42c1-9426-2d504fe103df None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:11:33.650300 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:11:33.662241 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edf98a1-b7d7-42c1-9426-2d504fe103df None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:11:33.663707 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edf98a1-b7d7-42c1-9426-2d504fe103df None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:11:33.757875 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 632/1255] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 24 17:11:33 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 17:11:33.764057 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ea35e3-cf19-4296-82f7-88d414c3c7a3 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:11:33.861639 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ea35e3-cf19-4296-82f7-88d414c3c7a3 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:11:33.875941 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:11:33.897839 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f325a3-158f-4f57-b78c-f6e893333977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:11:33.898493 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f325a3-158f-4f57-b78c-f6e893333977 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:11:33.898770 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f325a3-158f-4f57-b78c-f6e893333977 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:11:33.899078 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f325a3-158f-4f57-b78c-f6e893333977 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:11:33.908799 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f325a3-158f-4f57-b78c-f6e893333977 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:11:33.910206 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f325a3-158f-4f57-b78c-f6e893333977 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:11:33.924435 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-d7f325a3-158f-4f57-b78c-f6e893333977 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:11:33.924435 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:11:33.924435 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:11:33.924435 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:11:33.924435 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:11:33.924435 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:11:33.924435 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:11:33.924435 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:11:33.924435 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:11:33.924435 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:11:33.924435 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:11:33.924435 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:11:33.924435 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:11:33.924435 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:11:33.924435 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:11:33.924435 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:11:33.925527 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:11:33.925527 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:11:33.925527 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:11:33.925527 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:11:33.925527 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:11:33.925527 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:11:33.925527 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:11:33.925527 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 24 17:11:33.925527 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:11:33.925527 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:11:33.925527 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:11:33.925527 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:11:33.925527 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:11:33.925527 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:11:33.925527 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:11:33.925527 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:11:33.925527 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:11:33.926636 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:11:33.926636 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:11:33.926636 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:11:33.926636 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 624/1256] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 24 17:11:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:11:33.931374 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edf98a1-b7d7-42c1-9426-2d504fe103df None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:11:34.015366 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:11:34.026561 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edf98a1-b7d7-42c1-9426-2d504fe103df None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:11:34.060205 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43574f23-e778-475d-8e10-c35e9a23fc8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:11:34.060812 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43574f23-e778-475d-8e10-c35e9a23fc8e None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:11:34.061092 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43574f23-e778-475d-8e10-c35e9a23fc8e None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:11:34.062039 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43574f23-e778-475d-8e10-c35e9a23fc8e None admin] PATH_INFO: `/v3/domains` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:11:34.062690 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43574f23-e778-475d-8e10-c35e9a23fc8e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:11:34.063020 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43574f23-e778-475d-8e10-c35e9a23fc8e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:11:34.064367 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43574f23-e778-475d-8e10-c35e9a23fc8e None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:11:34.075191 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 633/1257] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 24 17:11:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:11:34.081029 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f325a3-158f-4f57-b78c-f6e893333977 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:11:34.179325 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f325a3-158f-4f57-b78c-f6e893333977 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:11:34.211376 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ad0fcf-ba30-40de-8648-841f26fca998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:11:34.212001 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ad0fcf-ba30-40de-8648-841f26fca998 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:11:34.212280 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ad0fcf-ba30-40de-8648-841f26fca998 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:11:34.212552 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ad0fcf-ba30-40de-8648-841f26fca998 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:11:34.219488 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ad0fcf-ba30-40de-8648-841f26fca998 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:11:34.228853 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ad0fcf-ba30-40de-8648-841f26fca998 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:11:34.231759 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-d3ad0fcf-ba30-40de-8648-841f26fca998 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:11:34.231759 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:11:34.231759 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:11:34.231759 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:11:34.231759 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:11:34.231759 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:11:34.231759 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:11:34.231759 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:11:34.231759 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:11:34.231759 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:11:34.231759 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:11:34.231759 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:11:34.231759 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:11:34.231759 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:11:34.231759 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:11:34.232817 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:11:34.232817 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:11:34.232817 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:11:34.232817 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:11:34.232817 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:11:34.232817 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:11:34.232817 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:11:34.232817 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:11:34.232817 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 24 17:11:34.232817 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:11:34.232817 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:11:34.232817 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:11:34.232817 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:11:34.232817 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:11:34.232817 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:11:34.232817 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:11:34.232817 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:11:34.234367 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:11:34.234367 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:11:34.234367 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:11:34.234367 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:11:34.234367 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:11:34.234367 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:11:34.234367 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:11:34.234367 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:11:34.234367 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 625/1258] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 24 17:11:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:11:34.239457 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43574f23-e778-475d-8e10-c35e9a23fc8e None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:11:34.337332 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43574f23-e778-475d-8e10-c35e9a23fc8e None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:11:34.370656 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1366a02e-b3c9-4892-8c55-470f6bd7918e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:11:34.371291 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1366a02e-b3c9-4892-8c55-470f6bd7918e None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:11:34.371565 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1366a02e-b3c9-4892-8c55-470f6bd7918e None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:11:34.371832 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1366a02e-b3c9-4892-8c55-470f6bd7918e None admin] PATH_INFO: `/v3/projects` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:11:34.372511 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1366a02e-b3c9-4892-8c55-470f6bd7918e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:11:34.372812 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1366a02e-b3c9-4892-8c55-470f6bd7918e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:11:34.374500 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1366a02e-b3c9-4892-8c55-470f6bd7918e None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:11:34.386882 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 634/1259] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Aug 24 17:11:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:11:34.392937 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ad0fcf-ba30-40de-8648-841f26fca998 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:11:34.419069 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:11:34.490938 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ad0fcf-ba30-40de-8648-841f26fca998 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:11:34.524311 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3be647b-fe96-41c5-a668-2efbc6f7dbc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:11:34.525136 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3be647b-fe96-41c5-a668-2efbc6f7dbc6 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:11:34.525403 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3be647b-fe96-41c5-a668-2efbc6f7dbc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:11:34.525677 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3be647b-fe96-41c5-a668-2efbc6f7dbc6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:11:34.526398 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3be647b-fe96-41c5-a668-2efbc6f7dbc6 None admin] RBAC: Adding query filter params (role.id=c5346bc54abf445f9900edf94264c850, scope.project.id=46ab158bc3644c888557781b0630b0b8, user.id=03fc77c6107e430f90f7b6f7cfd4ec02) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:11:34.526706 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3be647b-fe96-41c5-a668-2efbc6f7dbc6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:11:34.528168 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3be647b-fe96-41c5-a668-2efbc6f7dbc6 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:11:34.534115 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 626/1260] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 24 17:11:34 2022] GET /identity/v3/role_assignments?user.id=03fc77c6107e430f90f7b6f7cfd4ec02&scope.project.id=46ab158bc3644c888557781b0630b0b8&role.id=c5346bc54abf445f9900edf94264c850 => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:11:36.401011 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b921d33-fb68-4138-853f-5ad59160da7e None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:11:36.401605 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b921d33-fb68-4138-853f-5ad59160da7e None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:11:36.401605 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b921d33-fb68-4138-853f-5ad59160da7e None None] PATH_INFO: `/` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:11:36.402956 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 635/1261] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:11:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:11:36.408487 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feaa485-8425-4eb6-8a8e-04129507b2c3 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:11:36.408921 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feaa485-8425-4eb6-8a8e-04129507b2c3 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:11:36.409021 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feaa485-8425-4eb6-8a8e-04129507b2c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:11:36.427138 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-4feaa485-8425-4eb6-8a8e-04129507b2c3 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:11:36.431856 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-4feaa485-8425-4eb6-8a8e-04129507b2c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:11:36.493855 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 627/1262] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:11:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:11:36.760230 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b921d33-fb68-4138-853f-5ad59160da7e None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:11:36.771081 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:11:36.857382 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b921d33-fb68-4138-853f-5ad59160da7e None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:11:36.892646 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9015c75-0a4c-4b1b-9578-aee3c3086697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:11:36.893267 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9015c75-0a4c-4b1b-9578-aee3c3086697 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:11:36.893538 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9015c75-0a4c-4b1b-9578-aee3c3086697 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:11:36.893803 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9015c75-0a4c-4b1b-9578-aee3c3086697 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:11:36.903547 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9015c75-0a4c-4b1b-9578-aee3c3086697 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:11:36.908243 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-d9015c75-0a4c-4b1b-9578-aee3c3086697 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:11:36.908243 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:11:36.908243 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:11:36.908243 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:11:36.908243 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:11:36.908243 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:11:36.908243 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:11:36.908243 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:11:36.908243 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:11:36.908243 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:11:36.908243 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:11:36.908243 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:11:36.908243 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:11:36.908243 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:11:36.908243 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:11:36.908243 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:11:36.909175 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:11:36.909175 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:11:36.909175 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:11:36.909175 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:11:36.909175 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:11:36.909175 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:11:36.909175 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:11:36.909175 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:11:36.909175 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:11:36.909175 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:11:36.909175 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:11:36.909175 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application with Lock( Aug 24 17:11:36.909175 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:11:36.909175 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:11:36.909175 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:11:36.909175 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:11:36.910129 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:11:36.910129 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:11:36.910129 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:11:36.910129 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:11:36.910129 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:11:36.910129 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:11:36.910129 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:11:36.910129 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:11:36.910129 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:11:36.910129 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:11:36.910129 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:11:36.910129 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:11:36.910129 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 636/1263] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Aug 24 17:11:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:11:36.915245 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4feaa485-8425-4eb6-8a8e-04129507b2c3 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:11:36.933538 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:11:37.014317 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4feaa485-8425-4eb6-8a8e-04129507b2c3 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:11:37.048497 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1776e424-6801-4d6a-b849-2cc29b0c30f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:11:37.049084 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1776e424-6801-4d6a-b849-2cc29b0c30f9 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:11:37.049365 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1776e424-6801-4d6a-b849-2cc29b0c30f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:11:37.049638 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1776e424-6801-4d6a-b849-2cc29b0c30f9 None admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:11:37.050295 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1776e424-6801-4d6a-b849-2cc29b0c30f9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:11:37.050591 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1776e424-6801-4d6a-b849-2cc29b0c30f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:11:37.051975 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1776e424-6801-4d6a-b849-2cc29b0c30f9 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:11:37.060167 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 628/1264] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 24 17:11:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:11:37.066356 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9015c75-0a4c-4b1b-9578-aee3c3086697 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:11:37.132459 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:11:37.164865 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9015c75-0a4c-4b1b-9578-aee3c3086697 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:11:37.197202 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f98f648-be9b-4307-b8a0-f3159be8ed2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:11:37.197812 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f98f648-be9b-4307-b8a0-f3159be8ed2e None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:11:37.198085 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f98f648-be9b-4307-b8a0-f3159be8ed2e None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:11:37.198350 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f98f648-be9b-4307-b8a0-f3159be8ed2e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:11:37.209814 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f98f648-be9b-4307-b8a0-f3159be8ed2e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:11:37.211279 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f98f648-be9b-4307-b8a0-f3159be8ed2e None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:11:37.225366 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-0f98f648-be9b-4307-b8a0-f3159be8ed2e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:11:37.225366 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:11:37.225366 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:11:37.225366 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:11:37.225366 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:11:37.225366 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:11:37.225366 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:11:37.225366 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:11:37.225366 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:11:37.225366 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:11:37.225366 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:11:37.225366 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:11:37.225366 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:11:37.225366 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:11:37.225366 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:11:37.225366 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:11:37.226514 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:11:37.226514 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:11:37.226514 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:11:37.226514 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:11:37.226514 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:11:37.226514 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:11:37.226514 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:11:37.226514 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application with Lock( Aug 24 17:11:37.226514 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:11:37.226514 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:11:37.226514 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:11:37.226514 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:11:37.226514 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:11:37.226514 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:11:37.226514 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:11:37.226514 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:11:37.226514 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:11:37.227838 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:11:37.227838 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:11:37.227838 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:11:37.227838 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 637/1265] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 24 17:11:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:11:37.232319 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1776e424-6801-4d6a-b849-2cc29b0c30f9 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:11:37.312966 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:11:37.339521 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1776e424-6801-4d6a-b849-2cc29b0c30f9 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:11:37.371943 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733aa63f-005e-40e7-b5b0-7a044f8bfc2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:11:37.372646 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733aa63f-005e-40e7-b5b0-7a044f8bfc2a None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:11:37.372930 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733aa63f-005e-40e7-b5b0-7a044f8bfc2a None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:11:37.373199 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733aa63f-005e-40e7-b5b0-7a044f8bfc2a None admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:11:37.374483 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733aa63f-005e-40e7-b5b0-7a044f8bfc2a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:11:37.374653 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733aa63f-005e-40e7-b5b0-7a044f8bfc2a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:11:37.376043 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733aa63f-005e-40e7-b5b0-7a044f8bfc2a None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:11:37.388322 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 629/1266] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 24 17:11:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:11:37.394708 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f98f648-be9b-4307-b8a0-f3159be8ed2e None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:11:37.491664 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f98f648-be9b-4307-b8a0-f3159be8ed2e None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:11:37.511919 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:11:37.525668 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed86a01-b7e5-43fe-a05d-004cee2a44a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:11:37.526280 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed86a01-b7e5-43fe-a05d-004cee2a44a5 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:11:37.526560 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed86a01-b7e5-43fe-a05d-004cee2a44a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:11:37.526834 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed86a01-b7e5-43fe-a05d-004cee2a44a5 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:11:37.534181 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed86a01-b7e5-43fe-a05d-004cee2a44a5 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:11:37.535645 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed86a01-b7e5-43fe-a05d-004cee2a44a5 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:11:37.547051 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-0ed86a01-b7e5-43fe-a05d-004cee2a44a5 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 17:11:37.547051 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:11:37.547051 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:11:37.547051 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:11:37.547051 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:11:37.547051 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:11:37.547051 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:11:37.547051 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:11:37.547051 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:11:37.547051 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:11:37.547051 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:11:37.547051 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:11:37.547051 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:11:37.547051 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:11:37.547051 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:11:37.547051 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:11:37.548083 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:11:37.548083 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:11:37.548083 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:11:37.548083 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:11:37.548083 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:11:37.548083 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:11:37.548083 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:11:37.548083 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:11:37.548083 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:11:37.548083 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:11:37.548083 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:11:37.548083 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application with Lock( Aug 24 17:11:37.548083 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:11:37.548083 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:11:37.548083 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:11:37.548083 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:11:37.548083 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:11:37.548083 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:11:37.549033 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:11:37.549033 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:11:37.549033 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:11:37.549033 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:11:37.549033 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:11:37.549033 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:11:37.549033 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:11:37.549033 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:11:37.549033 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 17:11:37.549033 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:11:37.554369 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 638/1267] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 24 17:11:37 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:11:37.554487 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733aa63f-005e-40e7-b5b0-7a044f8bfc2a None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:11:37.663180 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733aa63f-005e-40e7-b5b0-7a044f8bfc2a None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:11:37.697258 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e674cc6d-a0b7-4fa4-afd4-be5bdaeb548e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:11:37.697860 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e674cc6d-a0b7-4fa4-afd4-be5bdaeb548e None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:11:37.698139 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e674cc6d-a0b7-4fa4-afd4-be5bdaeb548e None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:11:37.698408 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e674cc6d-a0b7-4fa4-afd4-be5bdaeb548e None admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:11:37.699155 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e674cc6d-a0b7-4fa4-afd4-be5bdaeb548e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:11:37.699455 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e674cc6d-a0b7-4fa4-afd4-be5bdaeb548e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:11:37.700869 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e674cc6d-a0b7-4fa4-afd4-be5bdaeb548e None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:11:37.709546 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:11:37.717961 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 630/1268] 10.222.0.18 () {56 vars in 1143 bytes} [Wed Aug 24 17:11:37 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:11:37.723892 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed86a01-b7e5-43fe-a05d-004cee2a44a5 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:11:37.825142 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed86a01-b7e5-43fe-a05d-004cee2a44a5 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:11:37.858867 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea055e6-8b6e-46e4-9698-b2fd71d7276e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:11:37.859528 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea055e6-8b6e-46e4-9698-b2fd71d7276e None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:11:37.859805 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea055e6-8b6e-46e4-9698-b2fd71d7276e None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:11:37.860072 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea055e6-8b6e-46e4-9698-b2fd71d7276e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:11:37.870298 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea055e6-8b6e-46e4-9698-b2fd71d7276e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:11:37.871742 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea055e6-8b6e-46e4-9698-b2fd71d7276e None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:11:37.885550 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-6ea055e6-8b6e-46e4-9698-b2fd71d7276e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:11:37.885550 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:11:37.885550 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:11:37.885550 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:11:37.885550 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:11:37.885550 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:11:37.885550 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:11:37.885550 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:11:37.885550 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:11:37.885550 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:11:37.885550 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:11:37.885550 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:11:37.885550 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:11:37.885550 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:11:37.885550 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:11:37.885550 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:11:37.888862 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:11:37.888862 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:11:37.888862 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:11:37.888862 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:11:37.888862 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:11:37.888862 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:11:37.888862 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:11:37.888862 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application with Lock( Aug 24 17:11:37.888862 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:11:37.888862 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:11:37.888862 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:11:37.888862 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:11:37.888862 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:11:37.888862 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:11:37.888862 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:11:37.888862 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:11:37.888862 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:11:37.889854 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:11:37.889854 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:11:37.889854 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:11:37.889854 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 639/1269] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 24 17:11:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:11:37.892705 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e674cc6d-a0b7-4fa4-afd4-be5bdaeb548e None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:11:37.992902 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e674cc6d-a0b7-4fa4-afd4-be5bdaeb548e None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:11:38.025657 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0094abd8-62bd-4b74-9032-395809a32977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:11:38.026253 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0094abd8-62bd-4b74-9032-395809a32977 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:11:38.026530 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0094abd8-62bd-4b74-9032-395809a32977 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:11:38.026798 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0094abd8-62bd-4b74-9032-395809a32977 None admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:11:38.027464 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0094abd8-62bd-4b74-9032-395809a32977 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:11:38.027766 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0094abd8-62bd-4b74-9032-395809a32977 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:11:38.029117 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0094abd8-62bd-4b74-9032-395809a32977 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:11:38.040612 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 631/1270] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 24 17:11:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:11:38.047165 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea055e6-8b6e-46e4-9698-b2fd71d7276e None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:11:38.078707 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:11:38.145227 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea055e6-8b6e-46e4-9698-b2fd71d7276e None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:11:38.179590 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70f067d-5293-43a2-8576-ce9661e90ddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:11:38.180209 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70f067d-5293-43a2-8576-ce9661e90ddd None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:11:38.180551 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70f067d-5293-43a2-8576-ce9661e90ddd None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:11:38.180820 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70f067d-5293-43a2-8576-ce9661e90ddd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:11:38.187521 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70f067d-5293-43a2-8576-ce9661e90ddd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:11:38.188943 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70f067d-5293-43a2-8576-ce9661e90ddd None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:11:38.200905 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-b70f067d-5293-43a2-8576-ce9661e90ddd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:11:38.200905 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:11:38.200905 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:11:38.200905 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:11:38.200905 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:11:38.200905 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:11:38.200905 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:11:38.200905 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:11:38.200905 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:11:38.200905 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:11:38.200905 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:11:38.200905 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:11:38.200905 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:11:38.200905 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:11:38.200905 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:11:38.202516 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:11:38.202516 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:11:38.202516 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:11:38.202516 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:11:38.202516 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:11:38.202516 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:11:38.202516 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:11:38.202516 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:11:38.202516 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application with Lock( Aug 24 17:11:38.202516 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:11:38.202516 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:11:38.202516 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:11:38.202516 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:11:38.202516 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:11:38.202516 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:11:38.202516 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:11:38.202516 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:11:38.203710 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:11:38.203710 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:11:38.203710 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:11:38.203710 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:11:38.203710 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:11:38.203710 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:11:38.203710 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:11:38.203710 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:11:38.203710 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 640/1271] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 24 17:11:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:11:38.209271 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0094abd8-62bd-4b74-9032-395809a32977 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:11:38.268075 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:11:38.309952 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0094abd8-62bd-4b74-9032-395809a32977 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:11:38.343929 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c61dff-fb71-4ae0-a807-cccb89d5cc2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:11:38.344533 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c61dff-fb71-4ae0-a807-cccb89d5cc2d None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:11:38.344813 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c61dff-fb71-4ae0-a807-cccb89d5cc2d None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:11:38.345089 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c61dff-fb71-4ae0-a807-cccb89d5cc2d None admin] PATH_INFO: `/v3/projects` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:11:38.345778 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c61dff-fb71-4ae0-a807-cccb89d5cc2d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:11:38.346087 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c61dff-fb71-4ae0-a807-cccb89d5cc2d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:11:38.347553 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c61dff-fb71-4ae0-a807-cccb89d5cc2d None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:11:38.359471 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 632/1272] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Aug 24 17:11:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:11:38.365543 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70f067d-5293-43a2-8576-ce9661e90ddd None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:11:38.456323 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:11:38.464609 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70f067d-5293-43a2-8576-ce9661e90ddd None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:11:38.496540 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96031e9-cccb-432a-8bfd-e3a42ee19f69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:11:38.497183 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96031e9-cccb-432a-8bfd-e3a42ee19f69 None admin] REQUEST_METHOD: `PUT` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:11:38.497456 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96031e9-cccb-432a-8bfd-e3a42ee19f69 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:11:38.497722 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96031e9-cccb-432a-8bfd-e3a42ee19f69 None admin] PATH_INFO: `/v3/projects/46ab158bc3644c888557781b0630b0b8/users/03fc77c6107e430f90f7b6f7cfd4ec02/roles/c5346bc54abf445f9900edf94264c850` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:11:38.499643 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96031e9-cccb-432a-8bfd-e3a42ee19f69 None admin] RBAC: Authorizing `identity:create_grant(project_id=46ab158bc3644c888557781b0630b0b8, user_id=03fc77c6107e430f90f7b6f7cfd4ec02, role_id=c5346bc54abf445f9900edf94264c850)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:11:38.501099 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96031e9-cccb-432a-8bfd-e3a42ee19f69 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:11:38.610968 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 641/1273] 10.222.0.18 () {58 vars in 1439 bytes} [Wed Aug 24 17:11:38 2022] PUT /identity/v3/projects/46ab158bc3644c888557781b0630b0b8/users/03fc77c6107e430f90f7b6f7cfd4ec02/roles/c5346bc54abf445f9900edf94264c850 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:11:40.578834 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984731de-5e29-40e7-95e2-21566f43ad11 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:11:40.579386 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984731de-5e29-40e7-95e2-21566f43ad11 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:11:40.579487 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984731de-5e29-40e7-95e2-21566f43ad11 None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:11:40.580809 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 633/1274] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:11:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:11:40.586502 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78ff021-860a-41b6-8b73-352b42082c81 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:11:40.586987 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78ff021-860a-41b6-8b73-352b42082c81 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:11:40.587120 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78ff021-860a-41b6-8b73-352b42082c81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:11:40.604997 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-f78ff021-860a-41b6-8b73-352b42082c81 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:11:40.606243 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-f78ff021-860a-41b6-8b73-352b42082c81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:11:40.673008 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 642/1275] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:11:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 17:11:40.940124 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984731de-5e29-40e7-95e2-21566f43ad11 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:11:41.038894 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984731de-5e29-40e7-95e2-21566f43ad11 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:11:41.055391 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:11:41.072099 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e6e6c7-ec8b-4919-98c9-66caff5e77b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:11:41.072714 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e6e6c7-ec8b-4919-98c9-66caff5e77b6 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:11:41.072988 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e6e6c7-ec8b-4919-98c9-66caff5e77b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:11:41.073257 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e6e6c7-ec8b-4919-98c9-66caff5e77b6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:11:41.083435 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e6e6c7-ec8b-4919-98c9-66caff5e77b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 17:11:41.088007 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-f1e6e6c7-ec8b-4919-98c9-66caff5e77b6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:11:41.088007 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:11:41.088007 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:11:41.088007 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:11:41.088007 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:11:41.088007 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:11:41.088007 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:11:41.088007 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:11:41.088007 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:11:41.088007 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:11:41.088007 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:11:41.088007 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:11:41.088007 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 17:11:41.088007 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 17:11:41.088007 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 17:11:41.088007 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 17:11:41.089137 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 17:11:41.089137 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 17:11:41.089137 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 17:11:41.089137 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 17:11:41.089137 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:11:41.089137 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:11:41.089137 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:11:41.089137 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:11:41.089137 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:11:41.089137 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:11:41.089137 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:11:41.089137 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 24 17:11:41.089137 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:11:41.089137 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:11:41.089137 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:11:41.089137 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:11:41.090199 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:11:41.090199 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:11:41.090199 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:11:41.090199 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:11:41.090199 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 17:11:41.090199 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 17:11:41.090199 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 17:11:41.090199 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 17:11:41.090199 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 17:11:41.090199 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 17:11:41.090199 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 17:11:41.090199 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:11:41.090199 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 634/1276] 10.222.0.18 () {56 vars in 1096 bytes} [Wed Aug 24 17:11:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:11:41.097839 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78ff021-860a-41b6-8b73-352b42082c81 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:11:41.173524 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:11:41.193396 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78ff021-860a-41b6-8b73-352b42082c81 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:11:41.228909 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b29cc9-51bb-4b2c-8b18-85a5cac428bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:11:41.229512 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b29cc9-51bb-4b2c-8b18-85a5cac428bb None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:11:41.229794 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b29cc9-51bb-4b2c-8b18-85a5cac428bb None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:11:41.230096 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b29cc9-51bb-4b2c-8b18-85a5cac428bb None admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:11:41.230753 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b29cc9-51bb-4b2c-8b18-85a5cac428bb None admin] RBAC: Adding query filter params (name=admin) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:11:41.231090 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b29cc9-51bb-4b2c-8b18-85a5cac428bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:11:41.232469 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b29cc9-51bb-4b2c-8b18-85a5cac428bb None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:11:41.241045 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 643/1277] 10.222.0.18 () {56 vars in 1099 bytes} [Wed Aug 24 17:11:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:11:41.246839 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e6e6c7-ec8b-4919-98c9-66caff5e77b6 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:11:41.346944 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e6e6c7-ec8b-4919-98c9-66caff5e77b6 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:11:41.384826 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaaf429d-cae0-4021-8f90-6d93567418e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:11:41.385481 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaf429d-cae0-4021-8f90-6d93567418e9 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:11:41.385762 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaf429d-cae0-4021-8f90-6d93567418e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:11:41.386039 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaf429d-cae0-4021-8f90-6d93567418e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:11:41.396399 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaaf429d-cae0-4021-8f90-6d93567418e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:11:41.397878 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaaf429d-cae0-4021-8f90-6d93567418e9 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:11:41.411555 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-aaaf429d-cae0-4021-8f90-6d93567418e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:11:41.411555 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:11:41.411555 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:11:41.411555 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:11:41.411555 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:11:41.411555 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:11:41.411555 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:11:41.411555 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:11:41.411555 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:11:41.411555 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:11:41.411555 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:11:41.411555 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:11:41.411555 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:11:41.411555 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:11:41.411555 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:11:41.411555 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:11:41.412687 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:11:41.412687 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:11:41.412687 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:11:41.412687 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:11:41.412687 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:11:41.412687 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:11:41.412687 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:11:41.412687 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 24 17:11:41.412687 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:11:41.412687 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:11:41.412687 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:11:41.412687 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:11:41.412687 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:11:41.412687 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:11:41.412687 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:11:41.412687 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:11:41.412687 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:11:41.413622 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:11:41.413622 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:11:41.413622 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:11:41.413622 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 635/1278] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 24 17:11:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 17:11:41.418697 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b29cc9-51bb-4b2c-8b18-85a5cac428bb None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:11:41.519082 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b29cc9-51bb-4b2c-8b18-85a5cac428bb None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:11:41.553832 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acd3c92-958f-4f05-b5ce-ac626276294f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:11:41.554306 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acd3c92-958f-4f05-b5ce-ac626276294f None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:11:41.554587 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acd3c92-958f-4f05-b5ce-ac626276294f None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:11:41.554859 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acd3c92-958f-4f05-b5ce-ac626276294f None admin] PATH_INFO: `/v3/domains` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:11:41.555545 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acd3c92-958f-4f05-b5ce-ac626276294f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:11:41.555848 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acd3c92-958f-4f05-b5ce-ac626276294f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:11:41.557198 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acd3c92-958f-4f05-b5ce-ac626276294f None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:11:41.570192 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 644/1279] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 24 17:11:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:11:41.575931 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaaf429d-cae0-4021-8f90-6d93567418e9 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:11:41.607615 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:11:41.676630 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaaf429d-cae0-4021-8f90-6d93567418e9 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:11:41.709150 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02919ee-5b57-47a1-b789-3e9a1c178cdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:11:41.709767 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02919ee-5b57-47a1-b789-3e9a1c178cdb None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:11:41.710071 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02919ee-5b57-47a1-b789-3e9a1c178cdb None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:11:41.710340 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02919ee-5b57-47a1-b789-3e9a1c178cdb None admin] PATH_INFO: `/v3/users/placement` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:11:41.718070 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02919ee-5b57-47a1-b789-3e9a1c178cdb None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:11:41.721848 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02919ee-5b57-47a1-b789-3e9a1c178cdb None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:11:41.730460 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-c02919ee-5b57-47a1-b789-3e9a1c178cdb None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 17:11:41.730460 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:11:41.730460 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:11:41.730460 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:11:41.730460 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:11:41.730460 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:11:41.730460 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:11:41.730460 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:11:41.730460 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:11:41.730460 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:11:41.730460 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:11:41.730460 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:11:41.730460 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 17:11:41.730460 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 17:11:41.730460 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 17:11:41.730460 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 17:11:41.731596 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:11:41.731596 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:11:41.731596 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 17:11:41.731596 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:11:41.731596 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 17:11:41.731596 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 17:11:41.731596 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:11:41.731596 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:11:41.731596 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:11:41.731596 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:11:41.731596 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:11:41.731596 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 24 17:11:41.731596 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:11:41.731596 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:11:41.731596 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:11:41.731596 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:11:41.731596 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:11:41.731596 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:11:41.732793 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:11:41.732793 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:11:41.732793 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 17:11:41.732793 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 17:11:41.732793 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 17:11:41.732793 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 17:11:41.732793 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 17:11:41.732793 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 17:11:41.732793 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 17:11:41.732793 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:11:41.732793 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 636/1280] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 24 17:11:41 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:11:41.738186 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acd3c92-958f-4f05-b5ce-ac626276294f None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:11:41.741961 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:11:41.835879 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acd3c92-958f-4f05-b5ce-ac626276294f None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:11:41.870438 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7c408f-d1ce-48d4-84fc-55d154d25355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:11:41.871101 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7c408f-d1ce-48d4-84fc-55d154d25355 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:11:41.871520 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7c408f-d1ce-48d4-84fc-55d154d25355 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:11:41.871702 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7c408f-d1ce-48d4-84fc-55d154d25355 None admin] PATH_INFO: `/v3/users` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:11:41.872440 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7c408f-d1ce-48d4-84fc-55d154d25355 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:11:41.872811 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7c408f-d1ce-48d4-84fc-55d154d25355 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:11:41.874397 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7c408f-d1ce-48d4-84fc-55d154d25355 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:11:41.896112 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 645/1281] 10.222.0.18 () {56 vars in 1143 bytes} [Wed Aug 24 17:11:41 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:11:41.901915 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02919ee-5b57-47a1-b789-3e9a1c178cdb None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:11:41.992723 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:11:42.002175 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02919ee-5b57-47a1-b789-3e9a1c178cdb None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:11:42.035529 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7101f9eb-6aeb-4e68-97ab-e01e3128961d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:11:42.036180 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7101f9eb-6aeb-4e68-97ab-e01e3128961d None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:11:42.036458 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7101f9eb-6aeb-4e68-97ab-e01e3128961d None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:11:42.036726 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7101f9eb-6aeb-4e68-97ab-e01e3128961d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:11:42.102394 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7101f9eb-6aeb-4e68-97ab-e01e3128961d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:11:42.103792 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7101f9eb-6aeb-4e68-97ab-e01e3128961d None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:11:42.189929 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:11:42.199728 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 637/1282] 10.222.0.18 () {58 vars in 1282 bytes} [Wed Aug 24 17:11:41 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 17:11:42.205840 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7c408f-d1ce-48d4-84fc-55d154d25355 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:11:42.270116 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:11:42.307557 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7c408f-d1ce-48d4-84fc-55d154d25355 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:11:42.339954 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3b4708-b691-4369-b6aa-1e326e239f1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:11:42.340573 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3b4708-b691-4369-b6aa-1e326e239f1a None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:11:42.340866 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3b4708-b691-4369-b6aa-1e326e239f1a None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:11:42.341337 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3b4708-b691-4369-b6aa-1e326e239f1a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:11:42.352296 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3b4708-b691-4369-b6aa-1e326e239f1a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:11:42.353797 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3b4708-b691-4369-b6aa-1e326e239f1a None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:11:42.368261 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-dd3b4708-b691-4369-b6aa-1e326e239f1a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:11:42.368261 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:11:42.368261 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:11:42.368261 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:11:42.368261 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:11:42.368261 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:11:42.368261 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:11:42.368261 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:11:42.368261 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:11:42.368261 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:11:42.368261 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:11:42.368261 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:11:42.368261 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 17:11:42.368261 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 17:11:42.368261 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 17:11:42.368261 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 17:11:42.369394 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:11:42.369394 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:11:42.369394 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:11:42.369394 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:11:42.369394 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:11:42.369394 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:11:42.369394 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:11:42.369394 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application with Lock( Aug 24 17:11:42.369394 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:11:42.369394 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:11:42.369394 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:11:42.369394 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:11:42.369394 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:11:42.369394 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:11:42.369394 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:11:42.369394 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:11:42.369394 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 17:11:42.370481 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 17:11:42.370481 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 17:11:42.370481 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:11:42.370481 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 646/1283] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 24 17:11:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:11:42.375156 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7101f9eb-6aeb-4e68-97ab-e01e3128961d None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:11:42.475787 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7101f9eb-6aeb-4e68-97ab-e01e3128961d None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:11:42.510501 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d97ce36-6328-4cc3-9d31-3f24b4ec28d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:11:42.511124 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d97ce36-6328-4cc3-9d31-3f24b4ec28d7 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:11:42.511407 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d97ce36-6328-4cc3-9d31-3f24b4ec28d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:11:42.511675 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d97ce36-6328-4cc3-9d31-3f24b4ec28d7 None admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:11:42.512307 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d97ce36-6328-4cc3-9d31-3f24b4ec28d7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:11:42.512607 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d97ce36-6328-4cc3-9d31-3f24b4ec28d7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:11:42.513963 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d97ce36-6328-4cc3-9d31-3f24b4ec28d7 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:11:42.524783 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 638/1284] 10.222.0.18 () {56 vars in 1109 bytes} [Wed Aug 24 17:11:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:11:42.531097 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3b4708-b691-4369-b6aa-1e326e239f1a None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:11:42.628902 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3b4708-b691-4369-b6aa-1e326e239f1a None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:11:42.641712 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:11:42.661759 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5522c6-e8e4-4388-a141-6d9419dc2427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:11:42.662375 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5522c6-e8e4-4388-a141-6d9419dc2427 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:11:42.662650 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5522c6-e8e4-4388-a141-6d9419dc2427 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:11:42.662929 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5522c6-e8e4-4388-a141-6d9419dc2427 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:11:42.669397 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5522c6-e8e4-4388-a141-6d9419dc2427 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:11:42.671419 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5522c6-e8e4-4388-a141-6d9419dc2427 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:11:42.683230 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application [None req-0e5522c6-e8e4-4388-a141-6d9419dc2427 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:11:42.683230 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:11:42.683230 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:11:42.683230 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:11:42.683230 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:11:42.683230 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:11:42.683230 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:11:42.683230 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:11:42.683230 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:11:42.683230 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:11:42.683230 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:11:42.683230 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:11:42.683230 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 17:11:42.683230 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 17:11:42.683230 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 17:11:42.684328 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 17:11:42.684328 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:11:42.684328 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:11:42.684328 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:11:42.684328 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:11:42.684328 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:11:42.684328 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:11:42.684328 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:11:42.684328 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application with Lock( Aug 24 17:11:42.684328 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:11:42.684328 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:11:42.684328 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:11:42.684328 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:11:42.684328 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:11:42.684328 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:11:42.684328 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:11:42.684328 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:11:42.685376 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 17:11:42.685376 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 17:11:42.685376 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 17:11:42.685376 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 17:11:42.685376 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 17:11:42.685376 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 17:11:42.685376 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 17:11:42.685376 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: ERROR keystone.server.flask.application Aug 24 17:11:42.685376 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 647/1285] 10.222.0.18 () {56 vars in 1111 bytes} [Wed Aug 24 17:11:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:11:42.690338 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d97ce36-6328-4cc3-9d31-3f24b4ec28d7 None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:11:42.740387 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:11:42.792511 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d97ce36-6328-4cc3-9d31-3f24b4ec28d7 None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:11:42.825182 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4956c5-4f89-45f1-b0ee-bb36a66687fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:11:42.825784 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4956c5-4f89-45f1-b0ee-bb36a66687fe None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:11:42.826071 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4956c5-4f89-45f1-b0ee-bb36a66687fe None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:11:42.826349 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4956c5-4f89-45f1-b0ee-bb36a66687fe None admin] PATH_INFO: `/v3/projects` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:11:42.827570 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4956c5-4f89-45f1-b0ee-bb36a66687fe None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:11:42.827875 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4956c5-4f89-45f1-b0ee-bb36a66687fe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:11:42.829296 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4956c5-4f89-45f1-b0ee-bb36a66687fe None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:11:42.841019 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 639/1286] 10.222.0.18 () {56 vars in 1148 bytes} [Wed Aug 24 17:11:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:11:42.847833 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5522c6-e8e4-4388-a141-6d9419dc2427 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:11:42.950260 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5522c6-e8e4-4388-a141-6d9419dc2427 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:11:42.983810 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5568ad-0ae7-4c56-a503-ccc17b166f82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:11:42.984462 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5568ad-0ae7-4c56-a503-ccc17b166f82 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:11:42.984740 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5568ad-0ae7-4c56-a503-ccc17b166f82 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:11:42.985007 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5568ad-0ae7-4c56-a503-ccc17b166f82 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:11:42.985715 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5568ad-0ae7-4c56-a503-ccc17b166f82 None admin] RBAC: Adding query filter params (role.id=c5346bc54abf445f9900edf94264c850, scope.project.id=46ab158bc3644c888557781b0630b0b8, user.id=03fc77c6107e430f90f7b6f7cfd4ec02) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:11:42.986017 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5568ad-0ae7-4c56-a503-ccc17b166f82 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:11:42.987447 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5568ad-0ae7-4c56-a503-ccc17b166f82 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:11:42.992967 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 648/1287] 10.222.0.18 () {56 vars in 1374 bytes} [Wed Aug 24 17:11:42 2022] GET /identity/v3/role_assignments?user.id=03fc77c6107e430f90f7b6f7cfd4ec02&scope.project.id=46ab158bc3644c888557781b0630b0b8&role.id=c5346bc54abf445f9900edf94264c850 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:11:44.946807 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c1903f-6117-418d-be36-5df14e7be466 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:11:44.947414 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c1903f-6117-418d-be36-5df14e7be466 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:11:44.947414 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c1903f-6117-418d-be36-5df14e7be466 None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:11:44.948767 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 640/1288] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:11:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:11:44.953989 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e144d956-176a-411b-84d6-9282d9e7a51a None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:11:44.954413 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e144d956-176a-411b-84d6-9282d9e7a51a None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:11:44.954522 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e144d956-176a-411b-84d6-9282d9e7a51a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:11:44.972900 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-e144d956-176a-411b-84d6-9282d9e7a51a None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:11:44.974079 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-e144d956-176a-411b-84d6-9282d9e7a51a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:11:45.003615 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:11:45.038382 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 649/1289] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:11:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:11:45.309444 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c1903f-6117-418d-be36-5df14e7be466 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:11:45.410323 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c1903f-6117-418d-be36-5df14e7be466 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:11:45.421814 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:11:45.444563 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28385da2-f978-4c0c-b809-0f910ef97b8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:11:45.445184 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28385da2-f978-4c0c-b809-0f910ef97b8e None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:11:45.445465 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28385da2-f978-4c0c-b809-0f910ef97b8e None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:11:45.445736 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28385da2-f978-4c0c-b809-0f910ef97b8e None admin] PATH_INFO: `/v3/services/placement` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:11:45.446310 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28385da2-f978-4c0c-b809-0f910ef97b8e None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:11:45.447704 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28385da2-f978-4c0c-b809-0f910ef97b8e None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:11:45.456154 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-28385da2-f978-4c0c-b809-0f910ef97b8e None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 17:11:45.456154 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:11:45.456154 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 17:11:45.456154 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 17:11:45.456154 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 17:11:45.456154 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 17:11:45.456154 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 17:11:45.456154 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:11:45.456154 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 17:11:45.456154 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:11:45.456154 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 17:11:45.456154 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:11:45.456154 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:11:45.456154 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:11:45.456154 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:11:45.456154 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:11:45.456154 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:11:45.457358 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:11:45.457358 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:11:45.457358 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:11:45.457358 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:11:45.457358 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:11:45.457358 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:11:45.457358 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 17:11:45.457358 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 17:11:45.457358 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 17:11:45.457358 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 17:11:45.457358 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:11:45.457358 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:11:45.457358 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:11:45.457358 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:11:45.457358 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:11:45.457358 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:11:45.458265 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:11:45.458265 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 24 17:11:45.458265 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:11:45.458265 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:11:45.458265 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:11:45.458265 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:11:45.458265 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:11:45.458265 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:11:45.458265 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:11:45.458265 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:11:45.458265 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 17:11:45.458265 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:11:45.458265 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 17:11:45.458265 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:11:45.459681 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 641/1290] 10.222.0.18 () {56 vars in 1117 bytes} [Wed Aug 24 17:11:45 2022] GET /identity/v3/services/placement => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:11:45.465775 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e144d956-176a-411b-84d6-9282d9e7a51a None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:11:45.563060 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e144d956-176a-411b-84d6-9282d9e7a51a None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:11:45.596700 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c21a91-7ac0-43f3-b5ab-f267483b39d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:11:45.597436 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c21a91-7ac0-43f3-b5ab-f267483b39d5 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:11:45.597730 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c21a91-7ac0-43f3-b5ab-f267483b39d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:11:45.597984 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c21a91-7ac0-43f3-b5ab-f267483b39d5 None admin] PATH_INFO: `/v3/services` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:11:45.598641 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c21a91-7ac0-43f3-b5ab-f267483b39d5 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:11:45.598962 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c21a91-7ac0-43f3-b5ab-f267483b39d5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:11:45.600317 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c21a91-7ac0-43f3-b5ab-f267483b39d5 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:11:45.605043 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 650/1291] 10.222.0.18 () {56 vars in 1116 bytes} [Wed Aug 24 17:11:45 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:11:45.611443 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28385da2-f978-4c0c-b809-0f910ef97b8e None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:11:45.709897 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28385da2-f978-4c0c-b809-0f910ef97b8e None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:11:45.742957 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec14bee3-552f-4204-a3ff-da6017d80fe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:11:45.743557 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec14bee3-552f-4204-a3ff-da6017d80fe6 None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:11:45.743843 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec14bee3-552f-4204-a3ff-da6017d80fe6 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:11:45.744136 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec14bee3-552f-4204-a3ff-da6017d80fe6 None admin] PATH_INFO: `/v3/services` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:11:45.744756 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec14bee3-552f-4204-a3ff-da6017d80fe6 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:11:45.745058 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec14bee3-552f-4204-a3ff-da6017d80fe6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:11:45.746422 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec14bee3-552f-4204-a3ff-da6017d80fe6 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:11:45.750841 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 642/1292] 10.222.0.18 () {56 vars in 1116 bytes} [Wed Aug 24 17:11:45 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:11:47.666363 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606384e9-3d41-4380-8a96-c65f2e4bbd01 None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:11:47.666973 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606384e9-3d41-4380-8a96-c65f2e4bbd01 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:11:47.666973 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606384e9-3d41-4380-8a96-c65f2e4bbd01 None None] PATH_INFO: `/` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:11:47.668345 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 651/1293] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:11:47 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:11:47.674923 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32241146-749d-4f12-88eb-7a092c361439 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:11:47.675393 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32241146-749d-4f12-88eb-7a092c361439 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:11:47.675486 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32241146-749d-4f12-88eb-7a092c361439 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:11:47.693600 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-32241146-749d-4f12-88eb-7a092c361439 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:11:47.694786 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-32241146-749d-4f12-88eb-7a092c361439 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:11:47.724115 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:11:47.760143 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 643/1294] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:11:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:11:48.036788 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606384e9-3d41-4380-8a96-c65f2e4bbd01 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:11:48.068170 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:11:48.136052 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606384e9-3d41-4380-8a96-c65f2e4bbd01 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:11:48.168205 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962b7716-171b-4ba8-9612-d1beac4fad25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:11:48.168811 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962b7716-171b-4ba8-9612-d1beac4fad25 None admin] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:11:48.169099 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962b7716-171b-4ba8-9612-d1beac4fad25 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:11:48.169365 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962b7716-171b-4ba8-9612-d1beac4fad25 None admin] PATH_INFO: `/v3/services` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:11:48.170015 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962b7716-171b-4ba8-9612-d1beac4fad25 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:11:48.171978 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962b7716-171b-4ba8-9612-d1beac4fad25 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:11:48.249753 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 652/1295] 10.222.0.18 () {60 vars in 1141 bytes} [Wed Aug 24 17:11:48 2022] POST /identity/v3/services => generated 246 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:11:50.164308 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2182c9-97e0-4719-ab5d-73b6fa4ff412 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:11:50.164837 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2182c9-97e0-4719-ab5d-73b6fa4ff412 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:11:50.164837 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2182c9-97e0-4719-ab5d-73b6fa4ff412 None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:11:50.166205 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 644/1296] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:11:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:11:50.171986 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c006debb-4dc3-43c9-a47c-184b9dde0574 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:11:50.172369 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c006debb-4dc3-43c9-a47c-184b9dde0574 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:11:50.172527 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c006debb-4dc3-43c9-a47c-184b9dde0574 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:11:50.190412 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-c006debb-4dc3-43c9-a47c-184b9dde0574 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:11:50.191597 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-c006debb-4dc3-43c9-a47c-184b9dde0574 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:11:50.261883 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 653/1297] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:11:50 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:11:50.535907 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2182c9-97e0-4719-ab5d-73b6fa4ff412 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:11:50.636667 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2182c9-97e0-4719-ab5d-73b6fa4ff412 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:11:50.670224 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02df9053-ceed-48f7-9f81-eed3e795018e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:11:50.670954 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02df9053-ceed-48f7-9f81-eed3e795018e None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:11:50.671232 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02df9053-ceed-48f7-9f81-eed3e795018e None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:11:50.672056 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02df9053-ceed-48f7-9f81-eed3e795018e None admin] PATH_INFO: `/v3/services/placement` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:11:50.672631 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02df9053-ceed-48f7-9f81-eed3e795018e None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:11:50.674021 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02df9053-ceed-48f7-9f81-eed3e795018e None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:11:50.682491 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-02df9053-ceed-48f7-9f81-eed3e795018e None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 17:11:50.682491 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:11:50.682491 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 17:11:50.682491 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 17:11:50.682491 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 17:11:50.682491 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 17:11:50.682491 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 17:11:50.682491 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:11:50.682491 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 17:11:50.682491 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:11:50.682491 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 17:11:50.682491 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:11:50.682491 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:11:50.682491 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:11:50.682491 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:11:50.682491 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:11:50.682491 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:11:50.683717 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:11:50.683717 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:11:50.683717 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:11:50.683717 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:11:50.683717 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:11:50.683717 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:11:50.683717 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 17:11:50.683717 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 17:11:50.683717 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 17:11:50.683717 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 17:11:50.683717 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:11:50.683717 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:11:50.683717 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:11:50.683717 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:11:50.683717 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:11:50.683717 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:11:50.684590 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:11:50.684590 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 24 17:11:50.684590 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:11:50.684590 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:11:50.684590 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:11:50.684590 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:11:50.684590 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:11:50.684590 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:11:50.684590 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:11:50.684590 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:11:50.684590 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 17:11:50.684590 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:11:50.684590 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 17:11:50.684590 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:11:50.684590 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 645/1298] 10.222.0.18 () {56 vars in 1117 bytes} [Wed Aug 24 17:11:50 2022] GET /identity/v3/services/placement => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:11:50.689757 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c006debb-4dc3-43c9-a47c-184b9dde0574 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:11:50.707971 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:11:50.786088 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c006debb-4dc3-43c9-a47c-184b9dde0574 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:11:50.818407 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9192dce0-ddcf-4d74-9c6c-5e451d270f78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:11:50.819033 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9192dce0-ddcf-4d74-9c6c-5e451d270f78 None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:11:50.819312 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9192dce0-ddcf-4d74-9c6c-5e451d270f78 None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:11:50.819583 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9192dce0-ddcf-4d74-9c6c-5e451d270f78 None admin] PATH_INFO: `/v3/services` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:11:50.820201 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9192dce0-ddcf-4d74-9c6c-5e451d270f78 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:11:50.820501 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9192dce0-ddcf-4d74-9c6c-5e451d270f78 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:11:50.821854 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9192dce0-ddcf-4d74-9c6c-5e451d270f78 None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:11:50.826834 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 654/1299] 10.222.0.18 () {56 vars in 1116 bytes} [Wed Aug 24 17:11:50 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:11:50.832864 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02df9053-ceed-48f7-9f81-eed3e795018e None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:11:50.866161 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:11:50.932339 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02df9053-ceed-48f7-9f81-eed3e795018e None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:11:50.965181 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46676b31-c5cc-4500-a1e3-37a7f9cfa7dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:11:50.965774 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46676b31-c5cc-4500-a1e3-37a7f9cfa7dc None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:11:50.966050 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46676b31-c5cc-4500-a1e3-37a7f9cfa7dc None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:11:50.966317 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46676b31-c5cc-4500-a1e3-37a7f9cfa7dc None admin] PATH_INFO: `/v3/endpoints` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:11:50.967031 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46676b31-c5cc-4500-a1e3-37a7f9cfa7dc None admin] RBAC: Adding query filter params (interface=public, service_id=4af317d2649e439196bfa661718eb309, region_id=RegionOne) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:11:50.967375 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46676b31-c5cc-4500-a1e3-37a7f9cfa7dc None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:11:50.969323 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46676b31-c5cc-4500-a1e3-37a7f9cfa7dc None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:11:50.974186 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 646/1300] 10.222.0.18 () {56 vars in 1251 bytes} [Wed Aug 24 17:11:50 2022] GET /identity/v3/endpoints?service_id=4af317d2649e439196bfa661718eb309&interface=public®ion_id=RegionOne => generated 194 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:11:50.979985 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9192dce0-ddcf-4d74-9c6c-5e451d270f78 None admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:11:51.058303 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:11:51.075433 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9192dce0-ddcf-4d74-9c6c-5e451d270f78 None admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:11:51.108807 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701ad778-b05a-415c-b926-ec6f8bb76bff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:11:51.109457 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701ad778-b05a-415c-b926-ec6f8bb76bff None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:11:51.109734 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701ad778-b05a-415c-b926-ec6f8bb76bff None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:11:51.110003 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701ad778-b05a-415c-b926-ec6f8bb76bff None admin] PATH_INFO: `/v3/services` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:11:51.110593 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701ad778-b05a-415c-b926-ec6f8bb76bff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:11:51.112100 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701ad778-b05a-415c-b926-ec6f8bb76bff None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:11:51.118857 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 655/1301] 10.222.0.18 () {56 vars in 1087 bytes} [Wed Aug 24 17:11:50 2022] GET /identity/v3/services => generated 2191 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:11:53.008369 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206ffc15-3d16-472b-a87d-79bb786c19af None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:11:53.008898 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206ffc15-3d16-472b-a87d-79bb786c19af None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:11:53.008898 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206ffc15-3d16-472b-a87d-79bb786c19af None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:11:53.010243 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 647/1302] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:11:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:11:53.016057 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0710a846-b96b-47c6-a122-06bafe8a4700 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:11:53.016464 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0710a846-b96b-47c6-a122-06bafe8a4700 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:11:53.016607 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0710a846-b96b-47c6-a122-06bafe8a4700 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:11:53.034272 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-0710a846-b96b-47c6-a122-06bafe8a4700 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:11:53.035487 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-0710a846-b96b-47c6-a122-06bafe8a4700 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:11:53.100411 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 656/1303] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:11:53 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 17:11:53.369863 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206ffc15-3d16-472b-a87d-79bb786c19af None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:11:53.470481 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:11:53.477324 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206ffc15-3d16-472b-a87d-79bb786c19af None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:11:53.511115 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b001e0-bd33-4605-a126-03abe60b720a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:11:53.511741 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b001e0-bd33-4605-a126-03abe60b720a None admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:11:53.512019 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b001e0-bd33-4605-a126-03abe60b720a None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:11:53.512289 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b001e0-bd33-4605-a126-03abe60b720a None admin] PATH_INFO: `/v3/services/placement` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:11:53.512870 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b001e0-bd33-4605-a126-03abe60b720a None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:11:53.514228 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b001e0-bd33-4605-a126-03abe60b720a None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:11:53.522583 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application [None req-d2b001e0-bd33-4605-a126-03abe60b720a None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 17:11:53.522583 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:11:53.522583 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 17:11:53.522583 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 17:11:53.522583 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 17:11:53.522583 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 17:11:53.522583 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 17:11:53.522583 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:11:53.522583 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 17:11:53.522583 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:11:53.522583 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 17:11:53.522583 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:11:53.522583 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 17:11:53.522583 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 17:11:53.522583 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 17:11:53.522583 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 17:11:53.522583 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 17:11:53.523873 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 17:11:53.523873 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 17:11:53.523873 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 17:11:53.523873 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 17:11:53.523873 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 17:11:53.523873 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 17:11:53.523873 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 17:11:53.523873 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 17:11:53.523873 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 17:11:53.523873 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 17:11:53.523873 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 17:11:53.523873 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 17:11:53.523873 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 17:11:53.523873 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 17:11:53.523873 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 17:11:53.523873 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 17:11:53.524901 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 17:11:53.524901 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application with Lock( Aug 24 17:11:53.524901 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 17:11:53.524901 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self._enter() Aug 24 17:11:53.524901 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 17:11:53.524901 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 17:11:53.524901 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 17:11:53.524901 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application return self.creator() Aug 24 17:11:53.524901 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 17:11:53.524901 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application created_value = creator( Aug 24 17:11:53.524901 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 17:11:53.524901 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 17:11:53.524901 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 17:11:53.524901 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: ERROR keystone.server.flask.application Aug 24 17:11:53.524901 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 648/1304] 10.222.0.18 () {56 vars in 1117 bytes} [Wed Aug 24 17:11:53 2022] GET /identity/v3/services/placement => generated 90 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 17:11:53.529891 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0710a846-b96b-47c6-a122-06bafe8a4700 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:11:53.605488 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:11:53.628386 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0710a846-b96b-47c6-a122-06bafe8a4700 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:11:53.662589 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb445d4-0431-4388-b2f3-1c9962a0ef9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:11:53.663225 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb445d4-0431-4388-b2f3-1c9962a0ef9e None admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:11:53.663507 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb445d4-0431-4388-b2f3-1c9962a0ef9e None admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:11:53.663777 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb445d4-0431-4388-b2f3-1c9962a0ef9e None admin] PATH_INFO: `/v3/services` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:11:53.664426 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb445d4-0431-4388-b2f3-1c9962a0ef9e None admin] RBAC: Adding query filter params (name=placement) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:11:53.664705 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb445d4-0431-4388-b2f3-1c9962a0ef9e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:11:53.666065 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb445d4-0431-4388-b2f3-1c9962a0ef9e None admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:11:53.671093 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 657/1305] 10.222.0.18 () {56 vars in 1116 bytes} [Wed Aug 24 17:11:53 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:11:53.677425 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b001e0-bd33-4605-a126-03abe60b720a None admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:11:53.775179 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b001e0-bd33-4605-a126-03abe60b720a None admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:11:53.808641 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e71785d-1b4b-4a8f-86f5-b928fe0ccdb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:11:53.809244 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e71785d-1b4b-4a8f-86f5-b928fe0ccdb4 None admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:11:53.809513 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e71785d-1b4b-4a8f-86f5-b928fe0ccdb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:11:53.809783 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e71785d-1b4b-4a8f-86f5-b928fe0ccdb4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:11:53.810442 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e71785d-1b4b-4a8f-86f5-b928fe0ccdb4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:11:53.811822 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e71785d-1b4b-4a8f-86f5-b928fe0ccdb4 None admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:11:53.889564 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 649/1306] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 24 17:11:53 2022] POST /identity/v3/endpoints => generated 331 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:12:03.668068 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=30 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:12:03.885048 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:17:41.493749 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3234276f-b043-4ce1-9c0f-cf95eb1212e3 None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:17:41.494350 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3234276f-b043-4ce1-9c0f-cf95eb1212e3 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:17:41.494350 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3234276f-b043-4ce1-9c0f-cf95eb1212e3 None None] PATH_INFO: `/` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:17:41.500967 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffb09cd-4364-4758-9b4e-9bcc0b8b57ee None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:17:41.619595 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffb09cd-4364-4758-9b4e-9bcc0b8b57ee None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:17:41.619595 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffb09cd-4364-4758-9b4e-9bcc0b8b57ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:17:41.620426 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 658/1307] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:17:41 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:17:41.620518 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-1ffb09cd-4364-4758-9b4e-9bcc0b8b57ee None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:17:41.620518 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-1ffb09cd-4364-4758-9b4e-9bcc0b8b57ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:17:41.713841 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 650/1308] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:17:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:17:42.013545 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83abc9dc-14f3-49a0-9467-b5ec371e6084 None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:17:42.013849 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83abc9dc-14f3-49a0-9467-b5ec371e6084 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:17:42.014095 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83abc9dc-14f3-49a0-9467-b5ec371e6084 None None] PATH_INFO: `/` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:17:42.015872 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 659/1309] 10.222.0.18 () {54 vars in 965 bytes} [Wed Aug 24 17:17:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:17:42.022362 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ac1121-45c7-43e9-aeaa-63522f9334e6 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:17:42.022647 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ac1121-45c7-43e9-aeaa-63522f9334e6 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:17:42.022944 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ac1121-45c7-43e9-aeaa-63522f9334e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:17:42.121108 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-c0ac1121-45c7-43e9-aeaa-63522f9334e6 None None] MFA Rules not processed for user `0e9ed4d47aef4140bfb2d3bb798cc559`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:17:42.122303 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-c0ac1121-45c7-43e9-aeaa-63522f9334e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:17:42.144951 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:17:42.202394 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 651/1310] 10.222.0.18 () {58 vars in 1062 bytes} [Wed Aug 24 17:17:42 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:17:42.212208 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83abc9dc-14f3-49a0-9467-b5ec371e6084 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:17:42.214447 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-83abc9dc-14f3-49a0-9467-b5ec371e6084 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:17:42.337793 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83abc9dc-14f3-49a0-9467-b5ec371e6084 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:17:42.420486 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ff8e54-7cfd-4aec-923e-2e0600820d64 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e9ed4d47aef4140bfb2d3bb798cc559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:17:42.421108 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ff8e54-7cfd-4aec-923e-2e0600820d64 service neutron] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:17:42.421384 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ff8e54-7cfd-4aec-923e-2e0600820d64 service neutron] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:17:42.421650 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ff8e54-7cfd-4aec-923e-2e0600820d64 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:17:42.423367 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-b5ff8e54-7cfd-4aec-923e-2e0600820d64 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:17:42.487614 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:17:42.554235 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ff8e54-7cfd-4aec-923e-2e0600820d64 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:17:42.557130 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ff8e54-7cfd-4aec-923e-2e0600820d64 service neutron] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:17:42.579079 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 660/1311] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:17:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:17:44.792704 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8340d06c-5be7-4183-a5d2-e94b20070925 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:17:44.793246 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8340d06c-5be7-4183-a5d2-e94b20070925 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:17:44.793246 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8340d06c-5be7-4183-a5d2-e94b20070925 None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:17:44.794609 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 652/1312] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:17:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:17:44.800411 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a50199-0f94-4d41-945f-5ff96024a264 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:17:44.800911 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a50199-0f94-4d41-945f-5ff96024a264 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:17:44.801019 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a50199-0f94-4d41-945f-5ff96024a264 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:17:44.889151 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-d1a50199-0f94-4d41-945f-5ff96024a264 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:17:44.890374 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-d1a50199-0f94-4d41-945f-5ff96024a264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:17:44.984830 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 661/1313] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:17:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:17:45.266248 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8340d06c-5be7-4183-a5d2-e94b20070925 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:17:45.267814 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-8340d06c-5be7-4183-a5d2-e94b20070925 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:17:45.370440 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8340d06c-5be7-4183-a5d2-e94b20070925 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:17:45.441164 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:17:45.443666 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9230f1b-69eb-49bb-b9b2-c2a8a8ded250 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e9ed4d47aef4140bfb2d3bb798cc559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:17:45.444270 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9230f1b-69eb-49bb-b9b2-c2a8a8ded250 service neutron] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:17:45.444551 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9230f1b-69eb-49bb-b9b2-c2a8a8ded250 service neutron] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:17:45.444825 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9230f1b-69eb-49bb-b9b2-c2a8a8ded250 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:17:45.447063 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-b9230f1b-69eb-49bb-b9b2-c2a8a8ded250 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:17:45.558122 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9230f1b-69eb-49bb-b9b2-c2a8a8ded250 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:17:45.559606 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9230f1b-69eb-49bb-b9b2-c2a8a8ded250 service neutron] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:17:45.577195 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 653/1314] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:17:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:17:48.021909 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5825750-2767-451b-9312-605c87d74969 None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:17:48.022356 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5825750-2767-451b-9312-605c87d74969 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:17:48.022446 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5825750-2767-451b-9312-605c87d74969 None None] PATH_INFO: `/` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:17:48.023840 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 662/1315] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:17:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:17:48.029707 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9cbada-2661-4d41-bcce-56ad48d73f01 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:17:48.030145 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9cbada-2661-4d41-bcce-56ad48d73f01 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:17:48.030244 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9cbada-2661-4d41-bcce-56ad48d73f01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:17:48.094657 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-4d9cbada-2661-4d41-bcce-56ad48d73f01 None None] MFA Rules not processed for user `bd42523e33c34e618cc8f1f75df2b677`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:17:48.095877 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-4d9cbada-2661-4d41-bcce-56ad48d73f01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:17:48.147903 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:17:48.191824 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 654/1316] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:17:48 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:17:48.480549 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5825750-2767-451b-9312-605c87d74969 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:17:48.482077 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-f5825750-2767-451b-9312-605c87d74969 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:17:48.492044 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:17:48.593554 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5825750-2767-451b-9312-605c87d74969 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:17:48.667004 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90e6abd-4beb-4242-849c-550d0fc47dbb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e9ed4d47aef4140bfb2d3bb798cc559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:17:48.667611 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90e6abd-4beb-4242-849c-550d0fc47dbb service neutron] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:17:48.667886 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90e6abd-4beb-4242-849c-550d0fc47dbb service neutron] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:17:48.668153 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90e6abd-4beb-4242-849c-550d0fc47dbb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:17:48.669744 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-f90e6abd-4beb-4242-849c-550d0fc47dbb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:17:48.730390 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:17:48.775883 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90e6abd-4beb-4242-849c-550d0fc47dbb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:17:48.777340 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90e6abd-4beb-4242-849c-550d0fc47dbb service neutron] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:17:48.795843 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 663/1317] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:17:48 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:17:51.450679 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fc8bb5-ec5a-414b-80b1-6b67abdcfd59 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:17:51.451223 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fc8bb5-ec5a-414b-80b1-6b67abdcfd59 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:17:51.451223 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fc8bb5-ec5a-414b-80b1-6b67abdcfd59 None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:17:51.452538 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 655/1318] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:17:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 17:17:51.457822 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365ac926-aa17-40d6-ba47-c4778791650b None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:17:51.458264 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365ac926-aa17-40d6-ba47-c4778791650b None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:17:51.458370 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365ac926-aa17-40d6-ba47-c4778791650b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:17:51.523240 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-365ac926-aa17-40d6-ba47-c4778791650b None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:17:51.524967 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-365ac926-aa17-40d6-ba47-c4778791650b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:17:51.622964 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 664/1319] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:17:51 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:17:51.908822 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fc8bb5-ec5a-414b-80b1-6b67abdcfd59 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:17:51.910374 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-13fc8bb5-ec5a-414b-80b1-6b67abdcfd59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:17:52.016819 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fc8bb5-ec5a-414b-80b1-6b67abdcfd59 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:17:52.088040 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714d26fe-9114-4a34-8626-876b6707216e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e9ed4d47aef4140bfb2d3bb798cc559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:17:52.088638 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714d26fe-9114-4a34-8626-876b6707216e service neutron] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:17:52.088919 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714d26fe-9114-4a34-8626-876b6707216e service neutron] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:17:52.089192 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714d26fe-9114-4a34-8626-876b6707216e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:17:52.090782 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-714d26fe-9114-4a34-8626-876b6707216e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:17:52.098051 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:17:52.198494 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714d26fe-9114-4a34-8626-876b6707216e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:17:52.199880 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714d26fe-9114-4a34-8626-876b6707216e service neutron] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:17:52.217291 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 656/1320] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:17:51 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:17:55.105050 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284038a1-7574-4f2c-af05-30022df7c8c1 None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:17:55.105617 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284038a1-7574-4f2c-af05-30022df7c8c1 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:17:55.105617 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284038a1-7574-4f2c-af05-30022df7c8c1 None None] PATH_INFO: `/` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:17:55.106936 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 665/1321] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:17:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:17:55.112056 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb55c250-fef5-4b3f-95d7-1902f9772692 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:17:55.112503 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb55c250-fef5-4b3f-95d7-1902f9772692 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:17:55.112628 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb55c250-fef5-4b3f-95d7-1902f9772692 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:17:55.178274 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-fb55c250-fef5-4b3f-95d7-1902f9772692 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:17:55.179509 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-fb55c250-fef5-4b3f-95d7-1902f9772692 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:17:55.237228 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:17:55.277290 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 657/1322] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:17:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:17:55.583235 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c860b702-8eba-401e-88e4-8ba3fdce6db0 None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:17:55.583503 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c860b702-8eba-401e-88e4-8ba3fdce6db0 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:17:55.583823 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c860b702-8eba-401e-88e4-8ba3fdce6db0 None None] PATH_INFO: `/` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:17:55.585346 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 666/1323] 10.222.0.18 () {54 vars in 965 bytes} [Wed Aug 24 17:17:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:17:55.592243 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b358af5-6b0d-4483-92a1-5611e00d6d6a None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:17:55.592530 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b358af5-6b0d-4483-92a1-5611e00d6d6a None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:17:55.597016 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b358af5-6b0d-4483-92a1-5611e00d6d6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:17:55.673726 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-9b358af5-6b0d-4483-92a1-5611e00d6d6a None None] MFA Rules not processed for user `0e9ed4d47aef4140bfb2d3bb798cc559`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:17:55.674952 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-9b358af5-6b0d-4483-92a1-5611e00d6d6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:17:55.750713 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 658/1324] 10.222.0.18 () {58 vars in 1062 bytes} [Wed Aug 24 17:17:55 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:17:55.760782 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c860b702-8eba-401e-88e4-8ba3fdce6db0 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:17:55.762414 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-c860b702-8eba-401e-88e4-8ba3fdce6db0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:17:55.781165 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:17:55.865626 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c860b702-8eba-401e-88e4-8ba3fdce6db0 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:17:55.944597 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3306ec54-ce5a-4c59-816d-c5e5f1690742 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e9ed4d47aef4140bfb2d3bb798cc559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:17:55.945632 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3306ec54-ce5a-4c59-816d-c5e5f1690742 service neutron] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:17:55.946783 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3306ec54-ce5a-4c59-816d-c5e5f1690742 service neutron] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:17:55.947470 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3306ec54-ce5a-4c59-816d-c5e5f1690742 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:17:55.950097 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-3306ec54-ce5a-4c59-816d-c5e5f1690742 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:17:56.022074 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:17:56.060663 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3306ec54-ce5a-4c59-816d-c5e5f1690742 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:17:56.062094 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3306ec54-ce5a-4c59-816d-c5e5f1690742 service neutron] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:17:56.079741 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 667/1325] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:17:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:17:58.294997 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c4e8c3-6148-4c6e-8d2d-05da2779b143 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:17:58.295536 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c4e8c3-6148-4c6e-8d2d-05da2779b143 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:17:58.295536 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c4e8c3-6148-4c6e-8d2d-05da2779b143 None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:17:58.296894 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 659/1326] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:17:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:17:58.302292 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0395ec2-209d-4a12-82a5-f226ebcac4bb None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:17:58.302751 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0395ec2-209d-4a12-82a5-f226ebcac4bb None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:17:58.303010 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0395ec2-209d-4a12-82a5-f226ebcac4bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:17:58.373846 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-c0395ec2-209d-4a12-82a5-f226ebcac4bb None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:17:58.375081 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-c0395ec2-209d-4a12-82a5-f226ebcac4bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:17:58.473063 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 668/1327] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:17:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:17:58.756214 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c4e8c3-6148-4c6e-8d2d-05da2779b143 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:17:58.757864 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-34c4e8c3-6148-4c6e-8d2d-05da2779b143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:17:58.818436 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:17:58.862120 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c4e8c3-6148-4c6e-8d2d-05da2779b143 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:17:58.935909 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3676c789-e902-46d5-baad-729234fb81db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e9ed4d47aef4140bfb2d3bb798cc559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:17:58.936514 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3676c789-e902-46d5-baad-729234fb81db service neutron] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:17:58.936787 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3676c789-e902-46d5-baad-729234fb81db service neutron] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:17:58.937052 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3676c789-e902-46d5-baad-729234fb81db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:17:58.938650 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-3676c789-e902-46d5-baad-729234fb81db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:17:59.053420 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:17:59.056400 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3676c789-e902-46d5-baad-729234fb81db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:17:59.057844 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3676c789-e902-46d5-baad-729234fb81db service neutron] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:17:59.075519 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 660/1328] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:17:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:18:01.050663 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b48a174-d492-4f23-b86e-b2fd5fe341ac None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:18:01.051182 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b48a174-d492-4f23-b86e-b2fd5fe341ac None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:18:01.051281 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b48a174-d492-4f23-b86e-b2fd5fe341ac None None] PATH_INFO: `/` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:18:01.052593 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 669/1329] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:18:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:18:01.057972 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5938957c-dffc-4ee4-84ec-878857c1b364 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:18:01.058388 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5938957c-dffc-4ee4-84ec-878857c1b364 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:18:01.058523 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5938957c-dffc-4ee4-84ec-878857c1b364 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:18:01.140776 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-5938957c-dffc-4ee4-84ec-878857c1b364 None None] MFA Rules not processed for user `bd42523e33c34e618cc8f1f75df2b677`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:18:01.141990 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-5938957c-dffc-4ee4-84ec-878857c1b364 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:18:01.272480 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 661/1330] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:18:01 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:18:01.566573 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b48a174-d492-4f23-b86e-b2fd5fe341ac None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:18:01.568264 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-7b48a174-d492-4f23-b86e-b2fd5fe341ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:18:01.600916 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:18:01.679737 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b48a174-d492-4f23-b86e-b2fd5fe341ac None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:18:01.752794 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf722a3-48b2-4ce7-a872-75b7a29fb1f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e9ed4d47aef4140bfb2d3bb798cc559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:18:01.753325 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf722a3-48b2-4ce7-a872-75b7a29fb1f1 service neutron] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:18:01.753601 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf722a3-48b2-4ce7-a872-75b7a29fb1f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:18:01.753869 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf722a3-48b2-4ce7-a872-75b7a29fb1f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:18:01.755513 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-bbf722a3-48b2-4ce7-a872-75b7a29fb1f1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:18:01.837149 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:18:01.870869 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf722a3-48b2-4ce7-a872-75b7a29fb1f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:18:01.872338 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf722a3-48b2-4ce7-a872-75b7a29fb1f1 service neutron] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:18:01.891786 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 670/1331] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:18:01 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:18:04.392148 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b686f9-9b76-4318-8459-bf03011a2d97 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:18:04.392718 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b686f9-9b76-4318-8459-bf03011a2d97 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:18:04.392718 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b686f9-9b76-4318-8459-bf03011a2d97 None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:18:04.394108 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 662/1332] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:18:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:18:04.399973 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62445052-02ae-4734-9785-5ebcf501793e None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:18:04.400400 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62445052-02ae-4734-9785-5ebcf501793e None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:18:04.400503 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62445052-02ae-4734-9785-5ebcf501793e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:18:04.464067 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-62445052-02ae-4734-9785-5ebcf501793e None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:18:04.465247 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-62445052-02ae-4734-9785-5ebcf501793e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:18:04.558877 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 671/1333] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:18:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:18:04.847513 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b686f9-9b76-4318-8459-bf03011a2d97 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:18:04.849038 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-46b686f9-9b76-4318-8459-bf03011a2d97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:18:04.918263 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:18:04.952199 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b686f9-9b76-4318-8459-bf03011a2d97 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:18:05.023363 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c364677-c08d-4c8d-88a4-f1f80d4a18ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e9ed4d47aef4140bfb2d3bb798cc559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:18:05.023986 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c364677-c08d-4c8d-88a4-f1f80d4a18ce service neutron] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:18:05.024262 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c364677-c08d-4c8d-88a4-f1f80d4a18ce service neutron] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:18:05.024532 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c364677-c08d-4c8d-88a4-f1f80d4a18ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:18:05.026110 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-6c364677-c08d-4c8d-88a4-f1f80d4a18ce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:18:05.132563 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c364677-c08d-4c8d-88a4-f1f80d4a18ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:18:05.134028 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c364677-c08d-4c8d-88a4-f1f80d4a18ce service neutron] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:18:05.144204 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:18:05.152124 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 663/1334] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:18:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:18:07.108483 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e12a6c8-be69-4db3-980a-0ecd0eceaf5e None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:18:07.109074 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e12a6c8-be69-4db3-980a-0ecd0eceaf5e None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:18:07.109074 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e12a6c8-be69-4db3-980a-0ecd0eceaf5e None None] PATH_INFO: `/` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:18:07.110373 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 672/1335] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:18:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:18:07.115520 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e8bdeb-0326-4d1c-94b3-4307ca1744a3 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:18:07.115902 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e8bdeb-0326-4d1c-94b3-4307ca1744a3 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:18:07.116065 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e8bdeb-0326-4d1c-94b3-4307ca1744a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:18:07.181236 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-60e8bdeb-0326-4d1c-94b3-4307ca1744a3 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:18:07.182415 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-60e8bdeb-0326-4d1c-94b3-4307ca1744a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:18:07.276636 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 664/1336] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:18:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:18:07.571842 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e12a6c8-be69-4db3-980a-0ecd0eceaf5e None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:18:07.573376 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-9e12a6c8-be69-4db3-980a-0ecd0eceaf5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:18:07.612887 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:18:07.677146 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e12a6c8-be69-4db3-980a-0ecd0eceaf5e None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:18:07.747934 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4912a7-657d-4374-a7f1-4cf2a32b9bfe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e9ed4d47aef4140bfb2d3bb798cc559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:18:07.748530 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4912a7-657d-4374-a7f1-4cf2a32b9bfe service neutron] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:18:07.748806 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4912a7-657d-4374-a7f1-4cf2a32b9bfe service neutron] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:18:07.749072 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4912a7-657d-4374-a7f1-4cf2a32b9bfe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:18:07.750640 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-5c4912a7-657d-4374-a7f1-4cf2a32b9bfe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:18:07.830722 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:18:07.856041 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4912a7-657d-4374-a7f1-4cf2a32b9bfe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:18:07.857451 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4912a7-657d-4374-a7f1-4cf2a32b9bfe service neutron] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:18:07.875009 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 673/1337] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:18:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:18:10.469658 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70e075a-4e77-4cea-9d50-2d26d54c4f6f None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:18:10.470221 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70e075a-4e77-4cea-9d50-2d26d54c4f6f None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:18:10.470221 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70e075a-4e77-4cea-9d50-2d26d54c4f6f None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:18:10.471599 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 665/1338] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:18:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:18:10.476791 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96aadb03-06ac-4714-b905-fe91383198e8 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:18:10.477227 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96aadb03-06ac-4714-b905-fe91383198e8 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:18:10.477366 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96aadb03-06ac-4714-b905-fe91383198e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:18:10.543332 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-96aadb03-06ac-4714-b905-fe91383198e8 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:18:10.544539 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-96aadb03-06ac-4714-b905-fe91383198e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:18:10.639055 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 674/1339] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:18:10 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:18:10.918555 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70e075a-4e77-4cea-9d50-2d26d54c4f6f None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:18:10.920192 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-a70e075a-4e77-4cea-9d50-2d26d54c4f6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:18:10.995821 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:18:11.023101 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70e075a-4e77-4cea-9d50-2d26d54c4f6f None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:18:11.094021 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e67e0f-4eaf-4649-90f6-3feaad56245f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e9ed4d47aef4140bfb2d3bb798cc559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:18:11.094625 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e67e0f-4eaf-4649-90f6-3feaad56245f service neutron] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:18:11.094935 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e67e0f-4eaf-4649-90f6-3feaad56245f service neutron] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:18:11.095201 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e67e0f-4eaf-4649-90f6-3feaad56245f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:18:11.096780 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-f4e67e0f-4eaf-4649-90f6-3feaad56245f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:18:11.204140 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e67e0f-4eaf-4649-90f6-3feaad56245f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:18:11.205569 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e67e0f-4eaf-4649-90f6-3feaad56245f service neutron] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:18:11.223191 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 666/1340] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:18:10 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:18:15.363205 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931a2285-0080-493c-a67f-13c10dd54d5a None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:18:15.363801 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931a2285-0080-493c-a67f-13c10dd54d5a None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:18:15.363801 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931a2285-0080-493c-a67f-13c10dd54d5a None None] PATH_INFO: `/` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:18:15.365110 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 675/1341] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:18:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:18:15.370968 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101c1c54-2719-4985-b7a7-a49ca5b1e224 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:18:15.371384 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101c1c54-2719-4985-b7a7-a49ca5b1e224 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:18:15.371520 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101c1c54-2719-4985-b7a7-a49ca5b1e224 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:18:15.376922 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:18:15.438500 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-101c1c54-2719-4985-b7a7-a49ca5b1e224 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:18:15.439667 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-101c1c54-2719-4985-b7a7-a49ca5b1e224 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:18:15.550182 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 667/1342] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:18:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:18:15.833751 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931a2285-0080-493c-a67f-13c10dd54d5a None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:18:15.835316 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-931a2285-0080-493c-a67f-13c10dd54d5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:18:15.883169 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:18:15.938336 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931a2285-0080-493c-a67f-13c10dd54d5a None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:18:16.013107 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9e8268-65c7-4111-8bbb-705efcc433c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e9ed4d47aef4140bfb2d3bb798cc559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:18:16.013717 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9e8268-65c7-4111-8bbb-705efcc433c4 service neutron] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:18:16.013993 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9e8268-65c7-4111-8bbb-705efcc433c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:18:16.014261 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9e8268-65c7-4111-8bbb-705efcc433c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:18:16.015877 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-1b9e8268-65c7-4111-8bbb-705efcc433c4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:18:16.107994 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:18:16.124319 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9e8268-65c7-4111-8bbb-705efcc433c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:18:16.125742 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9e8268-65c7-4111-8bbb-705efcc433c4 service neutron] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:18:16.144444 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 676/1343] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:18:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:18:18.995085 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b6cbd0-3eef-447c-8ca5-eb0402589b38 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:18:18.995743 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b6cbd0-3eef-447c-8ca5-eb0402589b38 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:18:18.995743 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b6cbd0-3eef-447c-8ca5-eb0402589b38 None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:18:18.997141 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 668/1344] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:18:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:18:19.002149 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159c8bcd-1d80-4ce0-bb94-e021756a2a20 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:18:19.002597 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159c8bcd-1d80-4ce0-bb94-e021756a2a20 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:18:19.002703 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159c8bcd-1d80-4ce0-bb94-e021756a2a20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:18:19.068132 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-159c8bcd-1d80-4ce0-bb94-e021756a2a20 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:18:19.069317 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-159c8bcd-1d80-4ce0-bb94-e021756a2a20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:18:19.166191 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 677/1345] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:18:19 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:18:19.445427 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b6cbd0-3eef-447c-8ca5-eb0402589b38 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:18:19.446991 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-03b6cbd0-3eef-447c-8ca5-eb0402589b38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:18:19.543102 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:18:19.562604 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b6cbd0-3eef-447c-8ca5-eb0402589b38 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:18:19.639373 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b51b343-a352-440f-9968-ae8599cd83a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e9ed4d47aef4140bfb2d3bb798cc559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:18:19.640019 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b51b343-a352-440f-9968-ae8599cd83a3 service neutron] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:18:19.640296 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b51b343-a352-440f-9968-ae8599cd83a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:18:19.640569 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b51b343-a352-440f-9968-ae8599cd83a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:18:19.642155 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-9b51b343-a352-440f-9968-ae8599cd83a3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:18:19.757930 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b51b343-a352-440f-9968-ae8599cd83a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:18:19.759464 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b51b343-a352-440f-9968-ae8599cd83a3 service neutron] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:18:19.779843 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 669/1346] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:18:19 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:18:23.845618 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209c2cd2-58c3-430a-bc05-81188a6eaca9 None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:18:23.846159 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209c2cd2-58c3-430a-bc05-81188a6eaca9 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:18:23.846159 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209c2cd2-58c3-430a-bc05-81188a6eaca9 None None] PATH_INFO: `/` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:18:23.847585 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 678/1347] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:18:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:18:23.853731 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e510dfc-2d7a-44ab-a68a-52de450bbf2a None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:18:23.854196 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e510dfc-2d7a-44ab-a68a-52de450bbf2a None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:18:23.854292 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e510dfc-2d7a-44ab-a68a-52de450bbf2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:18:23.872181 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:18:23.926154 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-4e510dfc-2d7a-44ab-a68a-52de450bbf2a None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:18:23.927328 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-4e510dfc-2d7a-44ab-a68a-52de450bbf2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:18:24.027837 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 670/1348] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:18:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:18:24.329702 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209c2cd2-58c3-430a-bc05-81188a6eaca9 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:18:24.331074 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-209c2cd2-58c3-430a-bc05-81188a6eaca9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:18:24.393456 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:18:24.438947 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209c2cd2-58c3-430a-bc05-81188a6eaca9 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:18:24.510467 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3086569-f8c9-4682-9ff9-f5d0f5aef040 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e9ed4d47aef4140bfb2d3bb798cc559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:18:24.511456 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3086569-f8c9-4682-9ff9-f5d0f5aef040 service neutron] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:18:24.511738 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3086569-f8c9-4682-9ff9-f5d0f5aef040 service neutron] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:18:24.512008 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3086569-f8c9-4682-9ff9-f5d0f5aef040 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:18:24.513613 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-f3086569-f8c9-4682-9ff9-f5d0f5aef040 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:18:24.619201 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:18:24.623177 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3086569-f8c9-4682-9ff9-f5d0f5aef040 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:18:24.624633 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3086569-f8c9-4682-9ff9-f5d0f5aef040 service neutron] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:18:24.643423 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 679/1349] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:18:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:18:29.577381 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b5901a-22ea-49b9-acb6-7f9a6db1d4bf None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:18:29.577792 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b5901a-22ea-49b9-acb6-7f9a6db1d4bf None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:18:29.579908 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b5901a-22ea-49b9-acb6-7f9a6db1d4bf None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:18:29.579908 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 671/1350] 10.222.0.18 () {54 vars in 921 bytes} [Wed Aug 24 17:18:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:18:29.590215 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d6e579-22da-4899-97df-f6a5660b98a2 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:18:29.590215 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d6e579-22da-4899-97df-f6a5660b98a2 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:18:29.590215 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d6e579-22da-4899-97df-f6a5660b98a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:18:29.759887 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-a5d6e579-22da-4899-97df-f6a5660b98a2 None None] MFA Rules not processed for user `03fc77c6107e430f90f7b6f7cfd4ec02`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:18:29.761875 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-a5d6e579-22da-4899-97df-f6a5660b98a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:18:29.878378 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 680/1351] 10.222.0.18 () {58 vars in 1018 bytes} [Wed Aug 24 17:18:29 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:18:33.946915 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa68fc7-4056-430b-afb9-b871903c30ef None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:18:33.947456 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa68fc7-4056-430b-afb9-b871903c30ef None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:18:33.947456 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa68fc7-4056-430b-afb9-b871903c30ef None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:18:33.948978 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 672/1352] 10.222.0.18 () {54 vars in 921 bytes} [Wed Aug 24 17:18:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:18:33.953660 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d9612f-0ebb-4a68-827c-2bf1a5d91c5c None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:18:33.954485 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d9612f-0ebb-4a68-827c-2bf1a5d91c5c None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:18:33.954996 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d9612f-0ebb-4a68-827c-2bf1a5d91c5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:18:34.015097 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:18:34.025815 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=59 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:18:34.051854 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-97d9612f-0ebb-4a68-827c-2bf1a5d91c5c None None] MFA Rules not processed for user `03fc77c6107e430f90f7b6f7cfd4ec02`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:18:34.053678 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-97d9612f-0ebb-4a68-827c-2bf1a5d91c5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:18:34.167508 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 681/1353] 10.222.0.18 () {58 vars in 1018 bytes} [Wed Aug 24 17:18:33 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:18:35.618927 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de32629a-5347-4bd8-bc73-704a2031b556 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:18:35.619503 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de32629a-5347-4bd8-bc73-704a2031b556 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:18:35.619628 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de32629a-5347-4bd8-bc73-704a2031b556 None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:18:35.621071 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 673/1354] 10.222.0.18 () {54 vars in 921 bytes} [Wed Aug 24 17:18:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:18:35.626296 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a331f38-87ed-4a23-a62d-9656e6aad8e6 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:18:35.626862 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a331f38-87ed-4a23-a62d-9656e6aad8e6 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:18:35.626862 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a331f38-87ed-4a23-a62d-9656e6aad8e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:18:35.706159 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-8a331f38-87ed-4a23-a62d-9656e6aad8e6 None None] MFA Rules not processed for user `03fc77c6107e430f90f7b6f7cfd4ec02`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:18:35.707402 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-8a331f38-87ed-4a23-a62d-9656e6aad8e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:18:35.753554 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:18:35.819464 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 682/1355] 10.222.0.18 () {58 vars in 1018 bytes} [Wed Aug 24 17:18:35 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:18:39.141538 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e8077c-6d9c-46b8-9e89-06dac9aa4253 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:18:39.141986 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e8077c-6d9c-46b8-9e89-06dac9aa4253 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:18:39.141986 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e8077c-6d9c-46b8-9e89-06dac9aa4253 None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:18:39.156743 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 674/1356] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:18:39 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:18:39.164143 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a7e1d4-f5fa-4635-be19-e41f90de6c2e None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:18:39.164632 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a7e1d4-f5fa-4635-be19-e41f90de6c2e None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:18:39.164737 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a7e1d4-f5fa-4635-be19-e41f90de6c2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:18:39.244202 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-c0a7e1d4-f5fa-4635-be19-e41f90de6c2e None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:18:39.245682 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-c0a7e1d4-f5fa-4635-be19-e41f90de6c2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:18:39.365159 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 683/1357] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:18:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:18:39.667496 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a666350-1b35-4eb6-b39e-3ee3c0289b23 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:18:39.668057 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a666350-1b35-4eb6-b39e-3ee3c0289b23 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:18:39.668548 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a666350-1b35-4eb6-b39e-3ee3c0289b23 None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:18:39.670323 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 675/1358] 10.222.0.18 () {54 vars in 962 bytes} [Wed Aug 24 17:18:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:18:39.677146 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed49aab-97b8-453b-8f1d-326d6fddfb92 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:18:39.677440 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed49aab-97b8-453b-8f1d-326d6fddfb92 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:18:39.677849 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed49aab-97b8-453b-8f1d-326d6fddfb92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:18:39.694264 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:18:39.764915 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-7ed49aab-97b8-453b-8f1d-326d6fddfb92 None None] MFA Rules not processed for user `557aca9255cb4bbb8af52e796c5c5177`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:18:39.766128 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-7ed49aab-97b8-453b-8f1d-326d6fddfb92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:18:39.890128 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 684/1359] 10.222.0.18 () {58 vars in 1059 bytes} [Wed Aug 24 17:18:39 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:18:39.902581 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a666350-1b35-4eb6-b39e-3ee3c0289b23 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:18:39.904674 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-9a666350-1b35-4eb6-b39e-3ee3c0289b23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:18:40.061729 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a666350-1b35-4eb6-b39e-3ee3c0289b23 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:18:40.145128 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:18:40.185383 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e24eff-dfea-4c7b-991d-160190e02246 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557aca9255cb4bbb8af52e796c5c5177', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:18:40.185987 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e24eff-dfea-4c7b-991d-160190e02246 service nova] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:18:40.186260 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e24eff-dfea-4c7b-991d-160190e02246 service nova] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:18:40.186530 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e24eff-dfea-4c7b-991d-160190e02246 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:18:40.188125 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-d8e24eff-dfea-4c7b-991d-160190e02246 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:18:40.281164 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e24eff-dfea-4c7b-991d-160190e02246 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:18:40.282919 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e24eff-dfea-4c7b-991d-160190e02246 service nova] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:18:40.303572 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 676/1360] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:18:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:18:42.453674 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af48cfc-c536-4938-93b1-4f15ae0b4f2c None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:18:42.454263 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af48cfc-c536-4938-93b1-4f15ae0b4f2c None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:18:42.454573 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af48cfc-c536-4938-93b1-4f15ae0b4f2c None None] PATH_INFO: `/` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:18:42.455968 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 685/1361] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:18:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:18:42.461805 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db414644-ebf8-43a2-89e4-2aebc510959f None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:18:42.462318 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db414644-ebf8-43a2-89e4-2aebc510959f None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:18:42.462318 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db414644-ebf8-43a2-89e4-2aebc510959f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:18:42.551899 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-db414644-ebf8-43a2-89e4-2aebc510959f None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:18:42.554609 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-db414644-ebf8-43a2-89e4-2aebc510959f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:18:42.605613 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:18:42.670945 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 677/1362] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:18:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:18:42.971083 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af48cfc-c536-4938-93b1-4f15ae0b4f2c None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:18:42.972584 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-5af48cfc-c536-4938-93b1-4f15ae0b4f2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:18:43.058555 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:18:43.223595 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af48cfc-c536-4938-93b1-4f15ae0b4f2c None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:18:43.332634 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10715751-6a71-4d8a-a933-75d371f6f723 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557aca9255cb4bbb8af52e796c5c5177', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:18:43.333442 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10715751-6a71-4d8a-a933-75d371f6f723 service nova] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:18:43.333996 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10715751-6a71-4d8a-a933-75d371f6f723 service nova] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:18:43.334249 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10715751-6a71-4d8a-a933-75d371f6f723 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:18:43.337448 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-10715751-6a71-4d8a-a933-75d371f6f723 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:18:43.374980 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:18:43.460610 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10715751-6a71-4d8a-a933-75d371f6f723 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:18:43.463658 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10715751-6a71-4d8a-a933-75d371f6f723 service nova] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:18:43.491078 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 686/1363] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:18:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 520 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:18:45.942735 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c331c021-0414-49ce-9bbd-63b6f552b481 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:18:45.943364 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c331c021-0414-49ce-9bbd-63b6f552b481 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:18:45.943617 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c331c021-0414-49ce-9bbd-63b6f552b481 None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:18:45.945093 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 678/1364] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:18:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:18:45.950408 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bdc41a-a671-4ef1-9fe1-08034a03169a None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:18:45.950816 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bdc41a-a671-4ef1-9fe1-08034a03169a None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:18:45.951093 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bdc41a-a671-4ef1-9fe1-08034a03169a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:18:46.018908 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-12bdc41a-a671-4ef1-9fe1-08034a03169a None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:18:46.020256 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-12bdc41a-a671-4ef1-9fe1-08034a03169a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:18:46.128213 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:18:46.138766 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 687/1365] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:18:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:18:46.419804 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce95417-fa3b-4308-9ce2-3e4c3680246b None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:18:46.420212 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce95417-fa3b-4308-9ce2-3e4c3680246b None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:18:46.420586 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce95417-fa3b-4308-9ce2-3e4c3680246b None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:18:46.422094 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 679/1366] 10.222.0.18 () {54 vars in 962 bytes} [Wed Aug 24 17:18:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:18:46.427472 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539f0fd4-fbf1-44c8-b385-cd636c49eebf None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:18:46.427861 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539f0fd4-fbf1-44c8-b385-cd636c49eebf None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:18:46.428256 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539f0fd4-fbf1-44c8-b385-cd636c49eebf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:18:46.519688 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-539f0fd4-fbf1-44c8-b385-cd636c49eebf None None] MFA Rules not processed for user `557aca9255cb4bbb8af52e796c5c5177`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:18:46.521059 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-539f0fd4-fbf1-44c8-b385-cd636c49eebf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:18:46.653630 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 688/1367] 10.222.0.18 () {58 vars in 1059 bytes} [Wed Aug 24 17:18:46 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:18:46.661497 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce95417-fa3b-4308-9ce2-3e4c3680246b None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:18:46.663136 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-6ce95417-fa3b-4308-9ce2-3e4c3680246b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:18:46.826590 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce95417-fa3b-4308-9ce2-3e4c3680246b None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:18:46.847653 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:18:46.947630 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a79a55-990b-4c8d-8565-d9b55a71c3fa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557aca9255cb4bbb8af52e796c5c5177', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:18:46.948296 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a79a55-990b-4c8d-8565-d9b55a71c3fa service nova] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:18:46.948673 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a79a55-990b-4c8d-8565-d9b55a71c3fa service nova] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:18:46.949031 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a79a55-990b-4c8d-8565-d9b55a71c3fa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:18:46.950717 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-11a79a55-990b-4c8d-8565-d9b55a71c3fa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:18:47.042693 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a79a55-990b-4c8d-8565-d9b55a71c3fa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:18:47.044470 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a79a55-990b-4c8d-8565-d9b55a71c3fa service nova] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:18:47.063237 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 680/1368] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:18:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:18:47.342080 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0633bbc3-79b9-47d2-bab1-8baf1bf62845 None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:18:47.342625 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0633bbc3-79b9-47d2-bab1-8baf1bf62845 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:18:47.342625 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0633bbc3-79b9-47d2-bab1-8baf1bf62845 None None] PATH_INFO: `/` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:18:47.344313 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 689/1369] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:18:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 17:18:47.350119 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf8430d-1f32-41b0-9042-323bba988df4 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:18:47.350425 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf8430d-1f32-41b0-9042-323bba988df4 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:18:47.350681 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf8430d-1f32-41b0-9042-323bba988df4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:18:47.386781 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:18:47.427392 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-2bf8430d-1f32-41b0-9042-323bba988df4 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:18:47.428629 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-2bf8430d-1f32-41b0-9042-323bba988df4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:18:47.532024 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 681/1370] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:18:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:18:48.080175 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada135a7-2409-4bd2-b22b-abcda9f8e095 None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:18:48.080585 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada135a7-2409-4bd2-b22b-abcda9f8e095 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:18:48.081003 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada135a7-2409-4bd2-b22b-abcda9f8e095 None None] PATH_INFO: `/` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:18:48.082549 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 690/1371] 10.222.0.18 () {54 vars in 964 bytes} [Wed Aug 24 17:18:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:18:48.087961 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427170c6-69dc-4a8e-a01e-616f5db088bd None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:18:48.088420 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427170c6-69dc-4a8e-a01e-616f5db088bd None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:18:48.088511 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427170c6-69dc-4a8e-a01e-616f5db088bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:18:48.167105 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-427170c6-69dc-4a8e-a01e-616f5db088bd None None] MFA Rules not processed for user `be06246d47584b529150ceb98a5cfd95`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:18:48.168319 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-427170c6-69dc-4a8e-a01e-616f5db088bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:18:48.218805 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:18:48.252243 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 682/1372] 10.222.0.18 () {58 vars in 1061 bytes} [Wed Aug 24 17:18:48 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:18:48.260498 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada135a7-2409-4bd2-b22b-abcda9f8e095 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:18:48.262665 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-ada135a7-2409-4bd2-b22b-abcda9f8e095 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:18:48.315765 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:18:48.373788 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada135a7-2409-4bd2-b22b-abcda9f8e095 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:18:48.452187 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cc5a9a-89b0-4f52-88f1-f8890372b1e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be06246d47584b529150ceb98a5cfd95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:18:48.452802 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cc5a9a-89b0-4f52-88f1-f8890372b1e1 service cinder] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:18:48.453121 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cc5a9a-89b0-4f52-88f1-f8890372b1e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:18:48.453350 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cc5a9a-89b0-4f52-88f1-f8890372b1e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:18:48.454984 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-98cc5a9a-89b0-4f52-88f1-f8890372b1e1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:18:48.558289 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:18:48.572417 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cc5a9a-89b0-4f52-88f1-f8890372b1e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:18:48.574030 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cc5a9a-89b0-4f52-88f1-f8890372b1e1 service cinder] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:18:48.594559 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 691/1373] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:18:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:18:49.193163 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c8b6dd-2b73-4c30-93dc-b18bec57173d None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:18:49.193690 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c8b6dd-2b73-4c30-93dc-b18bec57173d None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:18:49.193690 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c8b6dd-2b73-4c30-93dc-b18bec57173d None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:18:49.195660 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 683/1374] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:18:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:18:49.200954 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eba0cca-cabc-4199-982a-d332180e2586 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:18:49.201243 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eba0cca-cabc-4199-982a-d332180e2586 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:18:49.201521 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eba0cca-cabc-4199-982a-d332180e2586 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:18:49.269815 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-4eba0cca-cabc-4199-982a-d332180e2586 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:18:49.271250 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-4eba0cca-cabc-4199-982a-d332180e2586 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:18:49.372014 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 692/1375] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:18:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:18:49.661263 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c8b6dd-2b73-4c30-93dc-b18bec57173d None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:18:49.662800 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-98c8b6dd-2b73-4c30-93dc-b18bec57173d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:18:49.810278 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c8b6dd-2b73-4c30-93dc-b18bec57173d None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:18:49.869734 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:18:49.931588 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588b25c3-11e1-4ef6-baf1-b518adf4fdaf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557aca9255cb4bbb8af52e796c5c5177', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:18:49.932116 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588b25c3-11e1-4ef6-baf1-b518adf4fdaf service nova] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:18:49.932394 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588b25c3-11e1-4ef6-baf1-b518adf4fdaf service nova] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:18:49.932662 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588b25c3-11e1-4ef6-baf1-b518adf4fdaf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:18:49.934242 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-588b25c3-11e1-4ef6-baf1-b518adf4fdaf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:18:50.025344 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588b25c3-11e1-4ef6-baf1-b518adf4fdaf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:18:50.026951 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588b25c3-11e1-4ef6-baf1-b518adf4fdaf service nova] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:18:50.045177 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 684/1376] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:18:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:18:52.101528 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddacc393-a188-4e4f-af25-23159bddeba4 None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:18:52.102039 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddacc393-a188-4e4f-af25-23159bddeba4 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:18:52.102458 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddacc393-a188-4e4f-af25-23159bddeba4 None None] PATH_INFO: `/` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:18:52.104724 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 693/1377] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:18:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:18:52.110197 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62043f2f-a840-438a-a21e-d1b14235d698 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:18:52.111019 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62043f2f-a840-438a-a21e-d1b14235d698 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:18:52.111374 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62043f2f-a840-438a-a21e-d1b14235d698 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:18:52.185086 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-62043f2f-a840-438a-a21e-d1b14235d698 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:18:52.186565 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-62043f2f-a840-438a-a21e-d1b14235d698 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:18:52.194363 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:18:52.316091 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 685/1378] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:18:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:18:52.615423 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddacc393-a188-4e4f-af25-23159bddeba4 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:18:52.617457 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-ddacc393-a188-4e4f-af25-23159bddeba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:18:52.683963 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:18:52.774681 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddacc393-a188-4e4f-af25-23159bddeba4 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:18:52.899441 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d29e863-31be-4a0b-9cd1-0abfb1f24255 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557aca9255cb4bbb8af52e796c5c5177', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:18:52.900126 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d29e863-31be-4a0b-9cd1-0abfb1f24255 service nova] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:18:52.900361 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d29e863-31be-4a0b-9cd1-0abfb1f24255 service nova] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:18:52.900591 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d29e863-31be-4a0b-9cd1-0abfb1f24255 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:18:52.902587 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-2d29e863-31be-4a0b-9cd1-0abfb1f24255 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:18:52.921953 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:18:53.000428 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d29e863-31be-4a0b-9cd1-0abfb1f24255 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:18:53.002043 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d29e863-31be-4a0b-9cd1-0abfb1f24255 service nova] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:18:53.021556 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 694/1379] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:18:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:18:55.116248 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520b3ca2-5a83-49da-8111-9f65f26e4334 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:18:55.117374 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520b3ca2-5a83-49da-8111-9f65f26e4334 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:18:55.117490 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520b3ca2-5a83-49da-8111-9f65f26e4334 None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:18:55.128218 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 686/1380] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:18:55 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:18:55.137436 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9886c84-3df0-4318-8fd4-36c8835c5bfc None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:18:55.138007 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9886c84-3df0-4318-8fd4-36c8835c5bfc None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:18:55.138007 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9886c84-3df0-4318-8fd4-36c8835c5bfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:18:55.217303 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-e9886c84-3df0-4318-8fd4-36c8835c5bfc None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:18:55.218510 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-e9886c84-3df0-4318-8fd4-36c8835c5bfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:18:55.320537 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:18:55.326443 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 695/1381] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:18:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:18:55.618118 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520b3ca2-5a83-49da-8111-9f65f26e4334 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:18:55.620363 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-520b3ca2-5a83-49da-8111-9f65f26e4334 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:18:55.778004 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:18:55.785694 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520b3ca2-5a83-49da-8111-9f65f26e4334 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:18:55.909249 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43be8cc2-8e9d-40f5-8a1b-0bf8b6cefdda service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557aca9255cb4bbb8af52e796c5c5177', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:18:55.909859 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43be8cc2-8e9d-40f5-8a1b-0bf8b6cefdda service nova] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:18:55.910138 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43be8cc2-8e9d-40f5-8a1b-0bf8b6cefdda service nova] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:18:55.910410 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43be8cc2-8e9d-40f5-8a1b-0bf8b6cefdda service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:18:55.912051 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-43be8cc2-8e9d-40f5-8a1b-0bf8b6cefdda service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:18:56.010220 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43be8cc2-8e9d-40f5-8a1b-0bf8b6cefdda service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:18:56.012112 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43be8cc2-8e9d-40f5-8a1b-0bf8b6cefdda service nova] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:18:56.023088 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:18:56.032160 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 687/1382] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:18:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:18:58.276180 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af33f8fa-e874-4170-8893-0eb32a20582e None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:18:58.276873 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af33f8fa-e874-4170-8893-0eb32a20582e None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:18:58.276873 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af33f8fa-e874-4170-8893-0eb32a20582e None None] PATH_INFO: `/` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:18:58.278238 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 696/1383] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:18:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 17:18:58.283668 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdb1614-ff00-4c81-acb9-fba54dcc8880 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:18:58.284136 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdb1614-ff00-4c81-acb9-fba54dcc8880 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:18:58.284333 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdb1614-ff00-4c81-acb9-fba54dcc8880 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:18:58.354406 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-4cdb1614-ff00-4c81-acb9-fba54dcc8880 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:18:58.355716 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-4cdb1614-ff00-4c81-acb9-fba54dcc8880 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:18:58.486681 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 688/1384] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:18:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:18:58.858713 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af33f8fa-e874-4170-8893-0eb32a20582e None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:18:58.860548 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-af33f8fa-e874-4170-8893-0eb32a20582e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:18:59.058008 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af33f8fa-e874-4170-8893-0eb32a20582e None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:18:59.128237 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:18:59.176979 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630f40dd-dcc0-4b79-9be6-cfc0e15899c8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557aca9255cb4bbb8af52e796c5c5177', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:18:59.177593 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630f40dd-dcc0-4b79-9be6-cfc0e15899c8 service nova] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:18:59.177871 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630f40dd-dcc0-4b79-9be6-cfc0e15899c8 service nova] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:18:59.178145 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630f40dd-dcc0-4b79-9be6-cfc0e15899c8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:18:59.179778 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-630f40dd-dcc0-4b79-9be6-cfc0e15899c8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:18:59.278979 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630f40dd-dcc0-4b79-9be6-cfc0e15899c8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:18:59.280680 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630f40dd-dcc0-4b79-9be6-cfc0e15899c8 service nova] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:18:59.299296 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 697/1385] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:18:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:19:01.372104 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57d54ce-d1cd-45f3-ad47-9648f314fabd None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:19:01.372697 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57d54ce-d1cd-45f3-ad47-9648f314fabd None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:19:01.372818 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57d54ce-d1cd-45f3-ad47-9648f314fabd None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:19:01.374336 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 689/1386] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:19:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:19:01.380197 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692dd1ac-5729-44d1-b6ed-2e185dbc2061 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:19:01.380635 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692dd1ac-5729-44d1-b6ed-2e185dbc2061 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:19:01.380822 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692dd1ac-5729-44d1-b6ed-2e185dbc2061 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:19:01.461333 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-692dd1ac-5729-44d1-b6ed-2e185dbc2061 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:19:01.463550 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-692dd1ac-5729-44d1-b6ed-2e185dbc2061 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:19:01.512687 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:19:01.581537 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 698/1387] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:19:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:19:01.876522 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57d54ce-d1cd-45f3-ad47-9648f314fabd None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:19:01.878181 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-e57d54ce-d1cd-45f3-ad47-9648f314fabd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:19:01.956835 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:19:02.050865 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57d54ce-d1cd-45f3-ad47-9648f314fabd None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:19:02.172492 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6deb954f-4f5b-41ce-98c5-4fadc2160196 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557aca9255cb4bbb8af52e796c5c5177', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:19:02.173042 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deb954f-4f5b-41ce-98c5-4fadc2160196 service nova] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:19:02.173325 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deb954f-4f5b-41ce-98c5-4fadc2160196 service nova] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:19:02.173605 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deb954f-4f5b-41ce-98c5-4fadc2160196 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:19:02.175307 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-6deb954f-4f5b-41ce-98c5-4fadc2160196 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:19:02.217125 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:19:02.273177 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6deb954f-4f5b-41ce-98c5-4fadc2160196 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:19:02.274872 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6deb954f-4f5b-41ce-98c5-4fadc2160196 service nova] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:19:02.295075 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 690/1388] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:19:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:19:04.134465 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d9e2b1-8cf0-46fa-9d4f-9604c378d739 None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:19:04.134984 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d9e2b1-8cf0-46fa-9d4f-9604c378d739 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:19:04.135175 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d9e2b1-8cf0-46fa-9d4f-9604c378d739 None None] PATH_INFO: `/` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:19:04.136801 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 699/1389] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:19:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:19:04.142107 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44200055-50cf-47a4-ac3a-1b14c57ec53f None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:19:04.142525 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44200055-50cf-47a4-ac3a-1b14c57ec53f None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:19:04.142658 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44200055-50cf-47a4-ac3a-1b14c57ec53f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:19:04.212550 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-44200055-50cf-47a4-ac3a-1b14c57ec53f None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:19:04.213754 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-44200055-50cf-47a4-ac3a-1b14c57ec53f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:19:04.311371 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 691/1390] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:19:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:19:04.353706 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc04207-c306-4d16-9e95-c1b206dd893e None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:19:04.353991 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc04207-c306-4d16-9e95-c1b206dd893e None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:19:04.354272 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc04207-c306-4d16-9e95-c1b206dd893e None None] PATH_INFO: `/` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:19:04.355661 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 700/1391] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:19:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:19:04.361071 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59820483-42f5-4085-978b-af1415b1324d None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:19:04.361355 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59820483-42f5-4085-978b-af1415b1324d None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:19:04.361627 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59820483-42f5-4085-978b-af1415b1324d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:19:04.377996 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:19:04.428449 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-59820483-42f5-4085-978b-af1415b1324d None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:19:04.430237 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-59820483-42f5-4085-978b-af1415b1324d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:19:04.533360 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 692/1392] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:19:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:19:04.621838 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963bac73-95b9-4198-bae5-c610b9d7bc71 None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:19:04.622219 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963bac73-95b9-4198-bae5-c610b9d7bc71 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:19:04.622560 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963bac73-95b9-4198-bae5-c610b9d7bc71 None None] PATH_INFO: `/` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:19:04.624023 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 701/1393] 10.222.0.18 () {54 vars in 967 bytes} [Wed Aug 24 17:19:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:19:04.629406 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd32d2b-06ab-4ac4-bf9b-31ae6e688e84 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:19:04.629638 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd32d2b-06ab-4ac4-bf9b-31ae6e688e84 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:19:04.629915 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd32d2b-06ab-4ac4-bf9b-31ae6e688e84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:19:04.713891 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-7bd32d2b-06ab-4ac4-bf9b-31ae6e688e84 None None] MFA Rules not processed for user `3553e522363046f6828fd61b2a0642ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:19:04.715112 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-7bd32d2b-06ab-4ac4-bf9b-31ae6e688e84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:19:04.749351 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:19:04.791696 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 693/1394] 10.222.0.18 () {58 vars in 1064 bytes} [Wed Aug 24 17:19:04 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:19:04.799351 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963bac73-95b9-4198-bae5-c610b9d7bc71 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:19:04.801162 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-963bac73-95b9-4198-bae5-c610b9d7bc71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:19:04.824645 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd32d2b-06ab-4ac4-bf9b-31ae6e688e84 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:19:04.825849 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-7bd32d2b-06ab-4ac4-bf9b-31ae6e688e84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:19:04.911406 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963bac73-95b9-4198-bae5-c610b9d7bc71 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:19:04.944574 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd32d2b-06ab-4ac4-bf9b-31ae6e688e84 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:19:04.969686 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:19:04.984937 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fb329d-29ce-430a-a2a7-01a49d6d7b10 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3553e522363046f6828fd61b2a0642ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:19:04.997078 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fb329d-29ce-430a-a2a7-01a49d6d7b10 service glance] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:19:04.997078 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fb329d-29ce-430a-a2a7-01a49d6d7b10 service glance] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:19:04.997078 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fb329d-29ce-430a-a2a7-01a49d6d7b10 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:19:04.997078 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-97fb329d-29ce-430a-a2a7-01a49d6d7b10 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:19:05.008480 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:19:05.059564 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653d843f-3128-4f0f-a055-6450d8b8f7eb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557aca9255cb4bbb8af52e796c5c5177', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:19:05.060170 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653d843f-3128-4f0f-a055-6450d8b8f7eb service nova] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:19:05.060447 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653d843f-3128-4f0f-a055-6450d8b8f7eb service nova] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:19:05.060719 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653d843f-3128-4f0f-a055-6450d8b8f7eb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:19:05.062301 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-653d843f-3128-4f0f-a055-6450d8b8f7eb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:19:05.096048 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fb329d-29ce-430a-a2a7-01a49d6d7b10 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:19:05.097476 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fb329d-29ce-430a-a2a7-01a49d6d7b10 service glance] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:19:05.116154 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 702/1395] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:19:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:19:05.150578 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653d843f-3128-4f0f-a055-6450d8b8f7eb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:19:05.152190 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653d843f-3128-4f0f-a055-6450d8b8f7eb service nova] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:19:05.170684 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 694/1396] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:19:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:19:06.014123 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9db1b7f-0a5a-4bd6-83fc-dbd18803d1b5 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:19:06.014640 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9db1b7f-0a5a-4bd6-83fc-dbd18803d1b5 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:19:06.014738 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9db1b7f-0a5a-4bd6-83fc-dbd18803d1b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:19:06.081232 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-c9db1b7f-0a5a-4bd6-83fc-dbd18803d1b5 None None] MFA Rules not processed for user `0ccc6723f7514e52863d102de3839951`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:19:06.082433 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-c9db1b7f-0a5a-4bd6-83fc-dbd18803d1b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:19:06.124922 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:19:06.171940 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 703/1397] 10.222.0.18 () {58 vars in 1009 bytes} [Wed Aug 24 17:19:06 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:19:06.214564 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430b7ead-4434-4b67-8329-c7add0a8b720 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:19:06.215068 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430b7ead-4434-4b67-8329-c7add0a8b720 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:19:06.215171 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430b7ead-4434-4b67-8329-c7add0a8b720 None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:19:06.216521 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 695/1398] 10.222.0.18 () {54 vars in 943 bytes} [Wed Aug 24 17:19:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:19:06.222969 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cf3990-ac0d-4f60-9da3-aaef459844b4 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:19:06.223248 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cf3990-ac0d-4f60-9da3-aaef459844b4 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:19:06.223520 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cf3990-ac0d-4f60-9da3-aaef459844b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:19:06.299820 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-12cf3990-ac0d-4f60-9da3-aaef459844b4 None None] MFA Rules not processed for user `ef8caa353be34499a58cd25b72ee4032`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:19:06.300920 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-12cf3990-ac0d-4f60-9da3-aaef459844b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:19:06.406509 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 704/1399] 10.222.0.18 () {58 vars in 1040 bytes} [Wed Aug 24 17:19:06 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:19:06.417499 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430b7ead-4434-4b67-8329-c7add0a8b720 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:19:06.419283 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-430b7ead-4434-4b67-8329-c7add0a8b720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:19:06.495607 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:19:06.561345 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430b7ead-4434-4b67-8329-c7add0a8b720 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:19:06.666439 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c825603-75d5-402c-ade0-7fd651f4aa48 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef8caa353be34499a58cd25b72ee4032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:19:06.667091 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c825603-75d5-402c-ade0-7fd651f4aa48 service swift] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:19:06.667367 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c825603-75d5-402c-ade0-7fd651f4aa48 service swift] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:19:06.667638 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c825603-75d5-402c-ade0-7fd651f4aa48 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:19:06.670188 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-4c825603-75d5-402c-ade0-7fd651f4aa48 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:19:06.732618 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:19:06.754684 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c825603-75d5-402c-ade0-7fd651f4aa48 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:19:06.756310 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c825603-75d5-402c-ade0-7fd651f4aa48 service swift] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:19:06.765251 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 696/1400] 10.222.0.18 () {60 vars in 1384 bytes} [Wed Aug 24 17:19:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 17:19:07.267696 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964f7df4-bac0-4891-b7cb-f6067b9f841e None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:19:07.268215 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964f7df4-bac0-4891-b7cb-f6067b9f841e None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:19:07.268215 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964f7df4-bac0-4891-b7cb-f6067b9f841e None None] PATH_INFO: `/` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:19:07.269572 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 705/1401] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:19:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:19:07.279265 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8545aaa5-fb65-4e5a-99fb-166c39774cb8 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:19:07.280117 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8545aaa5-fb65-4e5a-99fb-166c39774cb8 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:19:07.280397 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8545aaa5-fb65-4e5a-99fb-166c39774cb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:19:07.350684 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-8545aaa5-fb65-4e5a-99fb-166c39774cb8 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:19:07.351983 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-8545aaa5-fb65-4e5a-99fb-166c39774cb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:19:07.460954 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 697/1402] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:19:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:19:07.773893 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964f7df4-bac0-4891-b7cb-f6067b9f841e None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:19:07.775872 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-964f7df4-bac0-4891-b7cb-f6067b9f841e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:19:07.796630 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:19:07.949112 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964f7df4-bac0-4891-b7cb-f6067b9f841e None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:19:08.076541 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cd6237-a475-4cf7-a0f3-591dc6deea12 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557aca9255cb4bbb8af52e796c5c5177', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:19:08.077202 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cd6237-a475-4cf7-a0f3-591dc6deea12 service nova] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:19:08.077487 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cd6237-a475-4cf7-a0f3-591dc6deea12 service nova] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:19:08.077761 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cd6237-a475-4cf7-a0f3-591dc6deea12 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:19:08.079450 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-29cd6237-a475-4cf7-a0f3-591dc6deea12 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:19:08.194116 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cd6237-a475-4cf7-a0f3-591dc6deea12 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:19:08.195897 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cd6237-a475-4cf7-a0f3-591dc6deea12 service nova] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:19:08.215845 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 706/1403] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:19:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:19:08.248968 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:19:10.316702 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b03036-9bc2-4200-8139-e95451c2b4a2 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:19:10.317534 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b03036-9bc2-4200-8139-e95451c2b4a2 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:19:10.317534 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b03036-9bc2-4200-8139-e95451c2b4a2 None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:19:10.320174 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 698/1404] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:19:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 17:19:10.326484 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e61a6f2-9e75-4b93-b49e-6d795c9aa8e3 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:19:10.326484 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e61a6f2-9e75-4b93-b49e-6d795c9aa8e3 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:19:10.326484 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e61a6f2-9e75-4b93-b49e-6d795c9aa8e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:19:10.393211 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-6e61a6f2-9e75-4b93-b49e-6d795c9aa8e3 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:19:10.394436 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-6e61a6f2-9e75-4b93-b49e-6d795c9aa8e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:19:10.452128 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:19:10.516968 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 707/1405] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:19:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:19:10.822860 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b03036-9bc2-4200-8139-e95451c2b4a2 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:19:10.824554 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-e9b03036-9bc2-4200-8139-e95451c2b4a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:19:10.894832 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:19:10.997565 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b03036-9bc2-4200-8139-e95451c2b4a2 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:19:11.125220 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d652b17-5ca6-420c-8458-354ed02b4e5a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557aca9255cb4bbb8af52e796c5c5177', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:19:11.125220 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d652b17-5ca6-420c-8458-354ed02b4e5a service nova] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:19:11.125220 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d652b17-5ca6-420c-8458-354ed02b4e5a service nova] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:19:11.125220 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d652b17-5ca6-420c-8458-354ed02b4e5a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:19:11.126118 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-1d652b17-5ca6-420c-8458-354ed02b4e5a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:19:11.139973 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:19:11.233166 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d652b17-5ca6-420c-8458-354ed02b4e5a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:19:11.236094 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d652b17-5ca6-420c-8458-354ed02b4e5a service nova] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:19:11.254482 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 699/1406] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:19:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:19:13.334033 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de02b710-b76c-4f51-a612-7c568cf3c55a None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:19:13.334033 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de02b710-b76c-4f51-a612-7c568cf3c55a None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:19:13.334033 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de02b710-b76c-4f51-a612-7c568cf3c55a None None] PATH_INFO: `/` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:19:13.341018 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 708/1407] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:19:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:19:13.341343 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d20842-25ed-4ece-b9f6-6ade832c1bb3 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:19:13.341805 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d20842-25ed-4ece-b9f6-6ade832c1bb3 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:19:13.341909 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d20842-25ed-4ece-b9f6-6ade832c1bb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:19:13.421885 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-83d20842-25ed-4ece-b9f6-6ade832c1bb3 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:19:13.423115 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-83d20842-25ed-4ece-b9f6-6ade832c1bb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:19:13.548831 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:19:13.569414 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 700/1408] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:19:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:19:13.883881 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de02b710-b76c-4f51-a612-7c568cf3c55a None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:19:13.886089 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-de02b710-b76c-4f51-a612-7c568cf3c55a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:19:14.058608 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de02b710-b76c-4f51-a612-7c568cf3c55a None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:19:14.097474 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:19:14.201160 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17fcc33-778a-45f7-90ef-6b35ccdd8d33 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557aca9255cb4bbb8af52e796c5c5177', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:19:14.202092 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17fcc33-778a-45f7-90ef-6b35ccdd8d33 service nova] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:19:14.202548 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17fcc33-778a-45f7-90ef-6b35ccdd8d33 service nova] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:19:14.202820 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17fcc33-778a-45f7-90ef-6b35ccdd8d33 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:19:14.204648 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-e17fcc33-778a-45f7-90ef-6b35ccdd8d33 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:19:14.302954 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17fcc33-778a-45f7-90ef-6b35ccdd8d33 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:19:14.305154 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17fcc33-778a-45f7-90ef-6b35ccdd8d33 service nova] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:19:14.323595 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 709/1409] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:19:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:19:24.122633 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=4 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:19:24.320910 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=83 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:19:30.474058 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b454a004-a095-4fae-ad65-8dd471f7a2c7 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:19:30.474607 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b454a004-a095-4fae-ad65-8dd471f7a2c7 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:19:30.474607 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b454a004-a095-4fae-ad65-8dd471f7a2c7 None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:19:30.476627 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 701/1410] 10.222.0.18 () {54 vars in 918 bytes} [Wed Aug 24 17:19:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:19:30.481895 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42eb742-20bb-47b3-9578-3559bd4f94e7 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:19:30.482340 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42eb742-20bb-47b3-9578-3559bd4f94e7 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:19:30.482437 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42eb742-20bb-47b3-9578-3559bd4f94e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:19:30.557810 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-e42eb742-20bb-47b3-9578-3559bd4f94e7 None None] MFA Rules not processed for user `03fc77c6107e430f90f7b6f7cfd4ec02`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:19:30.559035 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-e42eb742-20bb-47b3-9578-3559bd4f94e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:19:30.667666 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 710/1411] 10.222.0.18 () {58 vars in 1015 bytes} [Wed Aug 24 17:19:30 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:19:30.681904 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08775c7b-ae0c-495e-8321-f48e5d37077f None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:19:30.682184 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08775c7b-ae0c-495e-8321-f48e5d37077f None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:19:30.682454 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08775c7b-ae0c-495e-8321-f48e5d37077f None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:19:30.683803 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 702/1412] 10.222.0.18 () {54 vars in 918 bytes} [Wed Aug 24 17:19:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:19:30.688328 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7598814d-fdec-430c-9635-c618558a0b5a None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:19:30.688615 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7598814d-fdec-430c-9635-c618558a0b5a None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:19:30.688913 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7598814d-fdec-430c-9635-c618558a0b5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:19:30.759753 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:19:30.762890 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-7598814d-fdec-430c-9635-c618558a0b5a None None] MFA Rules not processed for user `557aca9255cb4bbb8af52e796c5c5177`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:19:30.764111 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-7598814d-fdec-430c-9635-c618558a0b5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:19:30.878564 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 711/1413] 10.222.0.18 () {58 vars in 1015 bytes} [Wed Aug 24 17:19:30 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:19:33.251673 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc3ef0a-e3fb-475b-8a9b-6cb748c8bd00 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:19:33.252191 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc3ef0a-e3fb-475b-8a9b-6cb748c8bd00 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:19:33.252306 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc3ef0a-e3fb-475b-8a9b-6cb748c8bd00 None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:19:33.253587 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 703/1414] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 17:19:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:19:33.258922 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46d576a-a395-4de0-a570-5e1d311c0d7a None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:19:33.259359 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46d576a-a395-4de0-a570-5e1d311c0d7a None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:19:33.259463 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46d576a-a395-4de0-a570-5e1d311c0d7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:19:33.324204 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-d46d576a-a395-4de0-a570-5e1d311c0d7a None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:19:33.325397 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-d46d576a-a395-4de0-a570-5e1d311c0d7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:19:33.379138 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:19:33.421196 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 712/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 17:19:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:19:33.707583 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc3ef0a-e3fb-475b-8a9b-6cb748c8bd00 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:19:33.709132 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-3dc3ef0a-e3fb-475b-8a9b-6cb748c8bd00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:19:33.855146 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc3ef0a-e3fb-475b-8a9b-6cb748c8bd00 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:19:33.926087 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:19:33.966787 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60762773-9040-43bb-9390-3a621b17703b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557aca9255cb4bbb8af52e796c5c5177', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:19:33.967362 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60762773-9040-43bb-9390-3a621b17703b service nova] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:19:33.967638 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60762773-9040-43bb-9390-3a621b17703b service nova] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:19:33.967905 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60762773-9040-43bb-9390-3a621b17703b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:19:33.969506 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-60762773-9040-43bb-9390-3a621b17703b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:19:34.058474 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60762773-9040-43bb-9390-3a621b17703b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:19:34.060100 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60762773-9040-43bb-9390-3a621b17703b service nova] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:19:34.078289 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 704/1416] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:19:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:19:34.972472 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9587446d-2ee3-4ec0-a4af-5e41458846bf None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:19:34.973045 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9587446d-2ee3-4ec0-a4af-5e41458846bf None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:19:34.973045 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9587446d-2ee3-4ec0-a4af-5e41458846bf None None] PATH_INFO: `/` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:19:34.974380 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 713/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 17:19:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:19:34.979832 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63075213-d574-4837-9c9d-1340f538bf2f None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:19:34.980299 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63075213-d574-4837-9c9d-1340f538bf2f None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:19:34.980426 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63075213-d574-4837-9c9d-1340f538bf2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:19:35.055000 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-63075213-d574-4837-9c9d-1340f538bf2f None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:19:35.056200 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-63075213-d574-4837-9c9d-1340f538bf2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:19:35.082551 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:19:35.152071 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 705/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 17:19:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:19:35.299606 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9587446d-2ee3-4ec0-a4af-5e41458846bf None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:19:35.301039 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-9587446d-2ee3-4ec0-a4af-5e41458846bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:19:35.445885 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9587446d-2ee3-4ec0-a4af-5e41458846bf None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:19:35.485927 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:19:35.564777 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e706134d-d56f-42d3-b216-9e2a53c0bf10 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557aca9255cb4bbb8af52e796c5c5177', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:19:35.564777 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e706134d-d56f-42d3-b216-9e2a53c0bf10 service nova] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:19:35.564777 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e706134d-d56f-42d3-b216-9e2a53c0bf10 service nova] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:19:35.564777 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e706134d-d56f-42d3-b216-9e2a53c0bf10 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:19:35.565194 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-e706134d-d56f-42d3-b216-9e2a53c0bf10 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:19:35.663892 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e706134d-d56f-42d3-b216-9e2a53c0bf10 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:19:35.665567 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e706134d-d56f-42d3-b216-9e2a53c0bf10 service nova] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:19:35.683917 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 714/1419] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:19:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:19:36.555884 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf08f08a-f82e-419a-82f4-1e30d9fe6fe8 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:19:36.556380 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf08f08a-f82e-419a-82f4-1e30d9fe6fe8 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:19:36.556474 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf08f08a-f82e-419a-82f4-1e30d9fe6fe8 None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:19:36.557791 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 706/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 17:19:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:19:36.562768 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3338368c-9a1f-4dee-bd9c-64d8be649d90 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:19:36.563287 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3338368c-9a1f-4dee-bd9c-64d8be649d90 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:19:36.563419 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3338368c-9a1f-4dee-bd9c-64d8be649d90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:19:36.619363 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:19:36.641499 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-3338368c-9a1f-4dee-bd9c-64d8be649d90 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:19:36.642786 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-3338368c-9a1f-4dee-bd9c-64d8be649d90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:19:36.737818 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 715/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 17:19:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:19:36.890667 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf08f08a-f82e-419a-82f4-1e30d9fe6fe8 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:19:36.892252 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-cf08f08a-f82e-419a-82f4-1e30d9fe6fe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:19:37.038759 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf08f08a-f82e-419a-82f4-1e30d9fe6fe8 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:19:37.050811 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:19:37.151487 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caaf6d6-7300-49f3-af2b-5084393bb43a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557aca9255cb4bbb8af52e796c5c5177', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:19:37.152008 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caaf6d6-7300-49f3-af2b-5084393bb43a service nova] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:19:37.152286 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caaf6d6-7300-49f3-af2b-5084393bb43a service nova] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:19:37.152553 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caaf6d6-7300-49f3-af2b-5084393bb43a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:19:37.154137 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-0caaf6d6-7300-49f3-af2b-5084393bb43a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:19:37.242293 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0caaf6d6-7300-49f3-af2b-5084393bb43a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:19:37.243916 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0caaf6d6-7300-49f3-af2b-5084393bb43a service nova] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:19:37.262144 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 707/1422] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:19:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:19:38.116379 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2addf495-e2a6-4614-be49-0dad9743c625 None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:19:38.116935 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2addf495-e2a6-4614-be49-0dad9743c625 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:19:38.116935 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2addf495-e2a6-4614-be49-0dad9743c625 None None] PATH_INFO: `/` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:19:38.118352 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 716/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 17:19:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:19:38.123185 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7f9aef-cd7a-418d-9506-b0deba23c906 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:19:38.123634 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7f9aef-cd7a-418d-9506-b0deba23c906 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:19:38.123735 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7f9aef-cd7a-418d-9506-b0deba23c906 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:19:38.144085 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:19:38.199906 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-8b7f9aef-cd7a-418d-9506-b0deba23c906 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:19:38.201105 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-8b7f9aef-cd7a-418d-9506-b0deba23c906 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:19:38.296755 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 708/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 17:19:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:19:38.452892 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2addf495-e2a6-4614-be49-0dad9743c625 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:19:38.454302 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-2addf495-e2a6-4614-be49-0dad9743c625 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:19:38.567635 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:19:38.604498 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2addf495-e2a6-4614-be49-0dad9743c625 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:19:38.720870 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbf4388-865c-4a9c-a3e5-feea3f1e35f8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557aca9255cb4bbb8af52e796c5c5177', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:19:38.721390 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbf4388-865c-4a9c-a3e5-feea3f1e35f8 service nova] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:19:38.721663 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbf4388-865c-4a9c-a3e5-feea3f1e35f8 service nova] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:19:38.721930 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbf4388-865c-4a9c-a3e5-feea3f1e35f8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:19:38.723554 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-1bbf4388-865c-4a9c-a3e5-feea3f1e35f8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:19:38.797629 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:19:38.811193 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbf4388-865c-4a9c-a3e5-feea3f1e35f8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:19:38.812813 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbf4388-865c-4a9c-a3e5-feea3f1e35f8 service nova] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:19:38.830575 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 717/1425] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:19:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:19:39.696375 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fe70e1-10aa-4654-b6ef-658176fdf8dd None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:19:39.696960 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fe70e1-10aa-4654-b6ef-658176fdf8dd None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:19:39.696960 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fe70e1-10aa-4654-b6ef-658176fdf8dd None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:19:39.698293 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 709/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 17:19:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:19:39.703203 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cb27ab-3e7d-41da-945e-4d425fd93334 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:19:39.703607 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cb27ab-3e7d-41da-945e-4d425fd93334 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:19:39.703754 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cb27ab-3e7d-41da-945e-4d425fd93334 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:19:39.778105 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-75cb27ab-3e7d-41da-945e-4d425fd93334 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:19:39.783928 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-75cb27ab-3e7d-41da-945e-4d425fd93334 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:19:39.876703 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 718/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 17:19:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:19:40.030466 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fe70e1-10aa-4654-b6ef-658176fdf8dd None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:19:40.032011 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-d8fe70e1-10aa-4654-b6ef-658176fdf8dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:19:40.115883 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:19:40.178243 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fe70e1-10aa-4654-b6ef-658176fdf8dd None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:19:40.290297 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922e7a06-ff1a-4fcf-ada6-dd18a2857f96 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557aca9255cb4bbb8af52e796c5c5177', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:19:40.290921 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922e7a06-ff1a-4fcf-ada6-dd18a2857f96 service nova] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:19:40.291199 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922e7a06-ff1a-4fcf-ada6-dd18a2857f96 service nova] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:19:40.291466 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922e7a06-ff1a-4fcf-ada6-dd18a2857f96 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:19:40.293061 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-922e7a06-ff1a-4fcf-ada6-dd18a2857f96 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:19:40.340594 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:19:40.380133 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922e7a06-ff1a-4fcf-ada6-dd18a2857f96 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:19:40.382254 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922e7a06-ff1a-4fcf-ada6-dd18a2857f96 service nova] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:19:40.400779 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 710/1428] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:19:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:19:41.282076 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d67e57-f2b7-4bad-a1ef-629ede00e419 None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:19:41.282633 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d67e57-f2b7-4bad-a1ef-629ede00e419 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:19:41.282633 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d67e57-f2b7-4bad-a1ef-629ede00e419 None None] PATH_INFO: `/` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:19:41.284021 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 719/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 17:19:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:19:41.288875 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5502dca2-9d0f-43d9-a114-93dbe1fd716e None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:19:41.289322 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5502dca2-9d0f-43d9-a114-93dbe1fd716e None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:19:41.289522 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5502dca2-9d0f-43d9-a114-93dbe1fd716e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:19:41.364170 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-5502dca2-9d0f-43d9-a114-93dbe1fd716e None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:19:41.365400 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-5502dca2-9d0f-43d9-a114-93dbe1fd716e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:19:41.462762 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 711/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 17:19:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:19:41.611742 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d67e57-f2b7-4bad-a1ef-629ede00e419 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:19:41.613244 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-95d67e57-f2b7-4bad-a1ef-629ede00e419 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:19:41.665012 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:19:41.762133 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d67e57-f2b7-4bad-a1ef-629ede00e419 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:19:41.871778 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf2d2ae-9967-4d1a-8483-a8119cfc9069 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557aca9255cb4bbb8af52e796c5c5177', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:19:41.872382 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf2d2ae-9967-4d1a-8483-a8119cfc9069 service nova] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:19:41.872656 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf2d2ae-9967-4d1a-8483-a8119cfc9069 service nova] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:19:41.872922 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf2d2ae-9967-4d1a-8483-a8119cfc9069 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:19:41.874510 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-0cf2d2ae-9967-4d1a-8483-a8119cfc9069 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:19:41.881264 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:19:41.961430 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf2d2ae-9967-4d1a-8483-a8119cfc9069 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:19:41.963138 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf2d2ae-9967-4d1a-8483-a8119cfc9069 service nova] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:19:41.981460 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 720/1431] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:19:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:19:42.825039 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c284a5f3-c2cf-48a5-a4bd-7aa8ffe972ec None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:19:42.825591 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c284a5f3-c2cf-48a5-a4bd-7aa8ffe972ec None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:19:42.825591 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c284a5f3-c2cf-48a5-a4bd-7aa8ffe972ec None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:19:42.826957 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 712/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 17:19:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:19:42.831985 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c1f54c-c3e7-4b37-822f-a27b15d3c86c None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:19:42.832374 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c1f54c-c3e7-4b37-822f-a27b15d3c86c None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:19:42.832573 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c1f54c-c3e7-4b37-822f-a27b15d3c86c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:19:42.907989 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-66c1f54c-c3e7-4b37-822f-a27b15d3c86c None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:19:42.909189 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-66c1f54c-c3e7-4b37-822f-a27b15d3c86c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:19:42.976238 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:19:43.005257 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 721/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 17:19:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:19:43.153511 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c284a5f3-c2cf-48a5-a4bd-7aa8ffe972ec None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:19:43.154984 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-c284a5f3-c2cf-48a5-a4bd-7aa8ffe972ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:19:43.172311 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:19:43.301417 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c284a5f3-c2cf-48a5-a4bd-7aa8ffe972ec None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:19:43.386592 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:19:43.412406 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f89257-50e8-43c6-b09f-19cd84c99ded service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557aca9255cb4bbb8af52e796c5c5177', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:19:43.413437 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f89257-50e8-43c6-b09f-19cd84c99ded service nova] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:19:43.414055 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f89257-50e8-43c6-b09f-19cd84c99ded service nova] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:19:43.414639 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f89257-50e8-43c6-b09f-19cd84c99ded service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:19:43.416612 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-c1f89257-50e8-43c6-b09f-19cd84c99ded service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:19:43.506321 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f89257-50e8-43c6-b09f-19cd84c99ded service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:19:43.508223 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f89257-50e8-43c6-b09f-19cd84c99ded service nova] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:19:43.528625 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 713/1434] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:19:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:19:44.501204 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e34ffe7-6769-4a2e-85c2-e905c5ac4a9a None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:19:44.501807 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e34ffe7-6769-4a2e-85c2-e905c5ac4a9a None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:19:44.501807 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e34ffe7-6769-4a2e-85c2-e905c5ac4a9a None None] PATH_INFO: `/` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:19:44.503170 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 722/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 17:19:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:19:44.508835 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b238a3c3-48a3-4f96-bf17-60d109d18620 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:19:44.509271 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b238a3c3-48a3-4f96-bf17-60d109d18620 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:19:44.509369 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b238a3c3-48a3-4f96-bf17-60d109d18620 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:19:44.591310 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-b238a3c3-48a3-4f96-bf17-60d109d18620 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:19:44.607532 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-b238a3c3-48a3-4f96-bf17-60d109d18620 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:19:44.638154 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:19:44.698697 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 714/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 17:19:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:19:44.850845 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e34ffe7-6769-4a2e-85c2-e905c5ac4a9a None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:19:44.852391 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-4e34ffe7-6769-4a2e-85c2-e905c5ac4a9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:19:45.006057 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e34ffe7-6769-4a2e-85c2-e905c5ac4a9a None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:19:45.059322 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:19:45.117402 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9161b995-7905-4669-98b6-da7d1d57c6f8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557aca9255cb4bbb8af52e796c5c5177', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:19:45.117952 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9161b995-7905-4669-98b6-da7d1d57c6f8 service nova] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:19:45.118226 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9161b995-7905-4669-98b6-da7d1d57c6f8 service nova] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:19:45.118491 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9161b995-7905-4669-98b6-da7d1d57c6f8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:19:45.120115 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-9161b995-7905-4669-98b6-da7d1d57c6f8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:19:45.207891 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9161b995-7905-4669-98b6-da7d1d57c6f8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:19:45.209520 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9161b995-7905-4669-98b6-da7d1d57c6f8 service nova] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:19:45.227133 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 723/1437] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:19:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:19:46.108996 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0360e186-6a61-4492-bf73-17e04ae555a8 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:19:46.109540 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0360e186-6a61-4492-bf73-17e04ae555a8 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:19:46.109540 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0360e186-6a61-4492-bf73-17e04ae555a8 None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:19:46.110819 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 715/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 17:19:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:19:46.116312 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab359126-8aeb-47ea-9534-4a79b92ba4f7 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:19:46.116765 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab359126-8aeb-47ea-9534-4a79b92ba4f7 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:19:46.116913 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab359126-8aeb-47ea-9534-4a79b92ba4f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:19:46.192631 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-ab359126-8aeb-47ea-9534-4a79b92ba4f7 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:19:46.194642 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-ab359126-8aeb-47ea-9534-4a79b92ba4f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:19:46.196645 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:19:46.294076 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 724/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 17:19:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:19:46.447668 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0360e186-6a61-4492-bf73-17e04ae555a8 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:19:46.449133 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-0360e186-6a61-4492-bf73-17e04ae555a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:19:46.597890 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0360e186-6a61-4492-bf73-17e04ae555a8 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:19:46.624012 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:19:46.711831 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a052c9f-f5b7-424a-bfab-70d74516156d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557aca9255cb4bbb8af52e796c5c5177', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:19:46.712350 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a052c9f-f5b7-424a-bfab-70d74516156d service nova] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:19:46.712629 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a052c9f-f5b7-424a-bfab-70d74516156d service nova] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:19:46.712900 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a052c9f-f5b7-424a-bfab-70d74516156d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:19:46.714510 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-4a052c9f-f5b7-424a-bfab-70d74516156d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:19:46.802838 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a052c9f-f5b7-424a-bfab-70d74516156d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:19:46.805011 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a052c9f-f5b7-424a-bfab-70d74516156d service nova] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:19:46.823233 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 716/1440] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:19:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:19:48.628370 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d881b46-1503-409a-a5af-9b660d0f950a None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:19:48.628903 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d881b46-1503-409a-a5af-9b660d0f950a None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:19:48.628903 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d881b46-1503-409a-a5af-9b660d0f950a None None] PATH_INFO: `/` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:19:48.630328 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 725/1441] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 17:19:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:19:48.635931 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65afba06-5cef-4697-98cf-7e067b760c1e None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:19:48.636355 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65afba06-5cef-4697-98cf-7e067b760c1e None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:19:48.636576 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65afba06-5cef-4697-98cf-7e067b760c1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:19:48.677434 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:19:48.702983 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-65afba06-5cef-4697-98cf-7e067b760c1e None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:19:48.704175 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-65afba06-5cef-4697-98cf-7e067b760c1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:19:48.800525 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 717/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 17:19:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:19:49.090133 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d881b46-1503-409a-a5af-9b660d0f950a None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:19:49.091726 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-9d881b46-1503-409a-a5af-9b660d0f950a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:19:49.195493 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d881b46-1503-409a-a5af-9b660d0f950a None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:19:49.227148 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:19:49.268772 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bbeaf8-0556-4d39-ba1e-ed1568a4fd90 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e9ed4d47aef4140bfb2d3bb798cc559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:19:49.269382 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bbeaf8-0556-4d39-ba1e-ed1568a4fd90 service neutron] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:19:49.269663 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bbeaf8-0556-4d39-ba1e-ed1568a4fd90 service neutron] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:19:49.269936 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bbeaf8-0556-4d39-ba1e-ed1568a4fd90 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:19:49.271573 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-c0bbeaf8-0556-4d39-ba1e-ed1568a4fd90 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:19:49.377430 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bbeaf8-0556-4d39-ba1e-ed1568a4fd90 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:19:49.378791 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bbeaf8-0556-4d39-ba1e-ed1568a4fd90 service neutron] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:19:49.397378 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 726/1443] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:19:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:19:51.451898 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b7f44e-b316-4628-9a79-d03fbe8573c5 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:19:51.452439 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b7f44e-b316-4628-9a79-d03fbe8573c5 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:19:51.452439 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b7f44e-b316-4628-9a79-d03fbe8573c5 None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:19:51.453794 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 718/1444] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 17:19:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:19:51.459140 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b5545f-d979-4c5c-af91-7cd1019b133f None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:19:51.460106 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b5545f-d979-4c5c-af91-7cd1019b133f None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:19:51.460210 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b5545f-d979-4c5c-af91-7cd1019b133f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:19:51.524789 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-51b5545f-d979-4c5c-af91-7cd1019b133f None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:19:51.525983 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-51b5545f-d979-4c5c-af91-7cd1019b133f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:19:51.539067 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:19:51.623550 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 727/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 17:19:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:19:51.908269 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b7f44e-b316-4628-9a79-d03fbe8573c5 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:19:51.909822 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-46b7f44e-b316-4628-9a79-d03fbe8573c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:19:52.011129 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b7f44e-b316-4628-9a79-d03fbe8573c5 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:19:52.023344 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:19:52.084112 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f805ca80-2d99-4c0c-9024-173521e34a25 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e9ed4d47aef4140bfb2d3bb798cc559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:19:52.084723 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f805ca80-2d99-4c0c-9024-173521e34a25 service neutron] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:19:52.085005 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f805ca80-2d99-4c0c-9024-173521e34a25 service neutron] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:19:52.085276 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f805ca80-2d99-4c0c-9024-173521e34a25 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:19:52.086882 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-f805ca80-2d99-4c0c-9024-173521e34a25 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:19:52.195914 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f805ca80-2d99-4c0c-9024-173521e34a25 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:19:52.197284 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f805ca80-2d99-4c0c-9024-173521e34a25 service neutron] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:19:52.214815 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 719/1446] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:19:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:19:54.569876 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc7861b-54f7-43d5-acfa-fa0feadd1586 None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:19:54.570371 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc7861b-54f7-43d5-acfa-fa0feadd1586 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:19:54.570477 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc7861b-54f7-43d5-acfa-fa0feadd1586 None None] PATH_INFO: `/` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:19:54.571810 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 728/1447] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 17:19:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:19:54.577225 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874aa41f-1a2d-4532-9027-b5b1346a6321 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:19:54.581380 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874aa41f-1a2d-4532-9027-b5b1346a6321 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:19:54.581380 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874aa41f-1a2d-4532-9027-b5b1346a6321 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:19:54.627496 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:19:54.645386 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-874aa41f-1a2d-4532-9027-b5b1346a6321 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:19:54.646588 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-874aa41f-1a2d-4532-9027-b5b1346a6321 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:19:54.742915 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 720/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 17:19:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:19:55.034803 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc7861b-54f7-43d5-acfa-fa0feadd1586 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:19:55.036434 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-ecc7861b-54f7-43d5-acfa-fa0feadd1586 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:19:55.138989 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc7861b-54f7-43d5-acfa-fa0feadd1586 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:19:55.180514 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:19:55.214438 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b70b83-f098-4bd7-9dac-955830bb29ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e9ed4d47aef4140bfb2d3bb798cc559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:19:55.215082 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b70b83-f098-4bd7-9dac-955830bb29ad service neutron] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:19:55.215365 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b70b83-f098-4bd7-9dac-955830bb29ad service neutron] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:19:55.215644 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b70b83-f098-4bd7-9dac-955830bb29ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:19:55.217240 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-f0b70b83-f098-4bd7-9dac-955830bb29ad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:19:55.323963 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b70b83-f098-4bd7-9dac-955830bb29ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:19:55.325386 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b70b83-f098-4bd7-9dac-955830bb29ad service neutron] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:19:55.343402 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 729/1449] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:19:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:19:57.378949 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e2f9c2-9179-43e6-9944-2f2b8f40c564 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:19:57.379510 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e2f9c2-9179-43e6-9944-2f2b8f40c564 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:19:57.379510 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e2f9c2-9179-43e6-9944-2f2b8f40c564 None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:19:57.380905 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 721/1450] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 17:19:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:19:57.387307 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bf8fcc-f48d-4d06-a1c9-d71bb488f54a None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:19:57.387736 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bf8fcc-f48d-4d06-a1c9-d71bb488f54a None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:19:57.387871 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bf8fcc-f48d-4d06-a1c9-d71bb488f54a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:19:57.452986 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-11bf8fcc-f48d-4d06-a1c9-d71bb488f54a None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:19:57.454204 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-11bf8fcc-f48d-4d06-a1c9-d71bb488f54a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:19:57.474246 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:19:57.549793 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 730/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 17:19:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:19:57.833522 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e2f9c2-9179-43e6-9944-2f2b8f40c564 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:19:57.835116 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-f4e2f9c2-9179-43e6-9944-2f2b8f40c564 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:19:57.936162 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e2f9c2-9179-43e6-9944-2f2b8f40c564 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:19:57.955226 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:19:58.008255 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90fde42-b85f-4444-b5cd-7cf8160abbdc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e9ed4d47aef4140bfb2d3bb798cc559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:19:58.008858 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90fde42-b85f-4444-b5cd-7cf8160abbdc service neutron] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:19:58.009137 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90fde42-b85f-4444-b5cd-7cf8160abbdc service neutron] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:19:58.009407 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90fde42-b85f-4444-b5cd-7cf8160abbdc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:19:58.011068 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-c90fde42-b85f-4444-b5cd-7cf8160abbdc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:19:58.124549 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90fde42-b85f-4444-b5cd-7cf8160abbdc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:19:58.125901 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90fde42-b85f-4444-b5cd-7cf8160abbdc service neutron] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:19:58.143759 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 722/1452] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:19:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:20:00.108150 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc28d5fa-d62c-4cd4-82f1-c77c6bc4be15 None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:20:00.108678 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc28d5fa-d62c-4cd4-82f1-c77c6bc4be15 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:20:00.108678 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc28d5fa-d62c-4cd4-82f1-c77c6bc4be15 None None] PATH_INFO: `/` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:20:00.109975 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 731/1453] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 17:20:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:20:00.115160 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fc3f49-fece-408a-bb9c-73aa7ebef0e6 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:20:00.115610 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fc3f49-fece-408a-bb9c-73aa7ebef0e6 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:20:00.115721 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fc3f49-fece-408a-bb9c-73aa7ebef0e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:20:00.179330 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:20:00.182051 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-82fc3f49-fece-408a-bb9c-73aa7ebef0e6 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:20:00.183311 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-82fc3f49-fece-408a-bb9c-73aa7ebef0e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:20:00.278949 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 723/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 17:20:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:20:00.562361 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc28d5fa-d62c-4cd4-82f1-c77c6bc4be15 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:20:00.563975 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-cc28d5fa-d62c-4cd4-82f1-c77c6bc4be15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:20:00.667792 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc28d5fa-d62c-4cd4-82f1-c77c6bc4be15 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:20:00.715780 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:20:00.740026 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68bb0c2-169c-4aac-9a06-974ce0e43200 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e9ed4d47aef4140bfb2d3bb798cc559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:20:00.740680 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68bb0c2-169c-4aac-9a06-974ce0e43200 service neutron] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:20:00.740970 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68bb0c2-169c-4aac-9a06-974ce0e43200 service neutron] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:20:00.741228 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68bb0c2-169c-4aac-9a06-974ce0e43200 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:20:00.742840 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-f68bb0c2-169c-4aac-9a06-974ce0e43200 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:20:00.849699 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68bb0c2-169c-4aac-9a06-974ce0e43200 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:20:00.851256 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68bb0c2-169c-4aac-9a06-974ce0e43200 service neutron] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:20:00.869291 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 732/1455] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:20:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:20:02.824023 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9674718-c483-4a8f-be2c-a070fc7cf196 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:20:02.824542 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9674718-c483-4a8f-be2c-a070fc7cf196 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:20:02.824655 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9674718-c483-4a8f-be2c-a070fc7cf196 None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:20:02.825955 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 724/1456] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 17:20:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 17:20:02.831446 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7783910-a063-4a62-8f37-ba69c534a713 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:20:02.831867 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7783910-a063-4a62-8f37-ba69c534a713 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:20:02.832054 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7783910-a063-4a62-8f37-ba69c534a713 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:20:02.899055 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-e7783910-a063-4a62-8f37-ba69c534a713 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:20:02.900264 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-e7783910-a063-4a62-8f37-ba69c534a713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:20:02.928125 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:20:02.993820 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 733/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 17:20:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:20:03.279988 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9674718-c483-4a8f-be2c-a070fc7cf196 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:20:03.281593 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-e9674718-c483-4a8f-be2c-a070fc7cf196 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:20:03.384204 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9674718-c483-4a8f-be2c-a070fc7cf196 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:20:03.411749 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:20:03.456826 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b86c58a-111c-4c61-a950-5b014808de2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e9ed4d47aef4140bfb2d3bb798cc559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:20:03.457361 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b86c58a-111c-4c61-a950-5b014808de2f service neutron] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:20:03.457641 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b86c58a-111c-4c61-a950-5b014808de2f service neutron] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:20:03.457918 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b86c58a-111c-4c61-a950-5b014808de2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:20:03.459590 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-7b86c58a-111c-4c61-a950-5b014808de2f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:20:03.567173 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b86c58a-111c-4c61-a950-5b014808de2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:20:03.568614 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b86c58a-111c-4c61-a950-5b014808de2f service neutron] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:20:03.586820 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 725/1458] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:20:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:20:05.524827 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d349007d-5d07-41bc-b2cb-67ef4c87dc7b None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:20:05.525356 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d349007d-5d07-41bc-b2cb-67ef4c87dc7b None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:20:05.525356 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d349007d-5d07-41bc-b2cb-67ef4c87dc7b None None] PATH_INFO: `/` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:20:05.526643 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 734/1459] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 17:20:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:20:05.532055 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2ecac3-4d80-4015-8b24-47fb56c5bccf None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:20:05.532476 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2ecac3-4d80-4015-8b24-47fb56c5bccf None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:20:05.532599 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2ecac3-4d80-4015-8b24-47fb56c5bccf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:20:05.599389 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-3e2ecac3-4d80-4015-8b24-47fb56c5bccf None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:20:05.600591 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-3e2ecac3-4d80-4015-8b24-47fb56c5bccf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:20:05.609692 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:20:05.697045 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 726/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 17:20:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:20:05.982422 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d349007d-5d07-41bc-b2cb-67ef4c87dc7b None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:20:05.983995 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-d349007d-5d07-41bc-b2cb-67ef4c87dc7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:20:06.095067 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d349007d-5d07-41bc-b2cb-67ef4c87dc7b None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:20:06.150196 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:20:06.168135 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe527757-6e07-43f3-9d0a-4cec4114160c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e9ed4d47aef4140bfb2d3bb798cc559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:20:06.168743 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe527757-6e07-43f3-9d0a-4cec4114160c service neutron] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:20:06.169020 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe527757-6e07-43f3-9d0a-4cec4114160c service neutron] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:20:06.169291 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe527757-6e07-43f3-9d0a-4cec4114160c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:20:06.170978 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-fe527757-6e07-43f3-9d0a-4cec4114160c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:20:06.279306 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe527757-6e07-43f3-9d0a-4cec4114160c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:20:06.280659 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe527757-6e07-43f3-9d0a-4cec4114160c service neutron] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:20:06.300357 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 735/1461] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:20:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:20:08.236819 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b945ce-d455-4b33-a18f-2e99368d092b None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:20:08.237404 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b945ce-d455-4b33-a18f-2e99368d092b None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:20:08.237404 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b945ce-d455-4b33-a18f-2e99368d092b None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:20:08.238707 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 727/1462] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 17:20:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:20:08.244056 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01573e14-cf0b-4f3e-b311-b95d0c218e68 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:20:08.244487 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01573e14-cf0b-4f3e-b311-b95d0c218e68 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:20:08.244623 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01573e14-cf0b-4f3e-b311-b95d0c218e68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:20:08.309906 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-01573e14-cf0b-4f3e-b311-b95d0c218e68 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:20:08.311136 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-01573e14-cf0b-4f3e-b311-b95d0c218e68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:20:08.349076 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:20:08.407149 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 736/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 17:20:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:20:08.688963 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b945ce-d455-4b33-a18f-2e99368d092b None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:20:08.690515 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-d2b945ce-d455-4b33-a18f-2e99368d092b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:20:08.800757 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b945ce-d455-4b33-a18f-2e99368d092b None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:20:08.837632 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:20:08.873843 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f42b3a-65a9-432f-8ed9-61a79dd985a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e9ed4d47aef4140bfb2d3bb798cc559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:20:08.874450 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f42b3a-65a9-432f-8ed9-61a79dd985a4 service neutron] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:20:08.874729 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f42b3a-65a9-432f-8ed9-61a79dd985a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:20:08.875046 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f42b3a-65a9-432f-8ed9-61a79dd985a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:20:08.876648 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-c7f42b3a-65a9-432f-8ed9-61a79dd985a4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:20:08.982973 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f42b3a-65a9-432f-8ed9-61a79dd985a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:20:08.984402 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f42b3a-65a9-432f-8ed9-61a79dd985a4 service neutron] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:20:09.002962 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 728/1464] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:20:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:20:10.938729 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a76912-f4a4-4b5a-90de-168c3f0bccff None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:20:10.939283 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a76912-f4a4-4b5a-90de-168c3f0bccff None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:20:10.939410 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a76912-f4a4-4b5a-90de-168c3f0bccff None None] PATH_INFO: `/` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:20:10.940701 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 737/1465] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 17:20:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:20:10.946713 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4daaa29-0294-497b-a6a9-f4ee78e95455 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:20:10.947196 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4daaa29-0294-497b-a6a9-f4ee78e95455 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:20:10.947292 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4daaa29-0294-497b-a6a9-f4ee78e95455 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:20:11.014260 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-a4daaa29-0294-497b-a6a9-f4ee78e95455 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:20:11.015503 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-a4daaa29-0294-497b-a6a9-f4ee78e95455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:20:11.033289 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:20:11.112542 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 729/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 17:20:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:20:11.405407 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a76912-f4a4-4b5a-90de-168c3f0bccff None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:20:11.407031 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-42a76912-f4a4-4b5a-90de-168c3f0bccff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:20:11.512867 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a76912-f4a4-4b5a-90de-168c3f0bccff None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:20:11.578533 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:20:11.585915 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85981e0a-a657-4d4a-b64d-02918989e5ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e9ed4d47aef4140bfb2d3bb798cc559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:20:11.586520 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85981e0a-a657-4d4a-b64d-02918989e5ca service neutron] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:20:11.586797 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85981e0a-a657-4d4a-b64d-02918989e5ca service neutron] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:20:11.587092 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85981e0a-a657-4d4a-b64d-02918989e5ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:20:11.588684 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-85981e0a-a657-4d4a-b64d-02918989e5ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:20:11.694464 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85981e0a-a657-4d4a-b64d-02918989e5ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:20:11.695956 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85981e0a-a657-4d4a-b64d-02918989e5ca service neutron] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:20:11.714049 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 738/1467] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:20:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:20:13.657101 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe174f1-2de4-4f72-8d32-04bef89c75b5 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:20:13.657601 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe174f1-2de4-4f72-8d32-04bef89c75b5 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:20:13.657702 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe174f1-2de4-4f72-8d32-04bef89c75b5 None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:20:13.659127 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 730/1468] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 17:20:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:20:13.665229 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a2fcbd-6b7d-4cea-9967-615e61ca771e None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:20:13.665651 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a2fcbd-6b7d-4cea-9967-615e61ca771e None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:20:13.665772 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a2fcbd-6b7d-4cea-9967-615e61ca771e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:20:13.732581 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-d1a2fcbd-6b7d-4cea-9967-615e61ca771e None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:20:13.733790 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-d1a2fcbd-6b7d-4cea-9967-615e61ca771e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:20:13.781772 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:20:13.829872 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 739/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 17:20:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:20:14.113760 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe174f1-2de4-4f72-8d32-04bef89c75b5 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:20:14.115373 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-bfe174f1-2de4-4f72-8d32-04bef89c75b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:20:14.218170 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe174f1-2de4-4f72-8d32-04bef89c75b5 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:20:14.264364 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:20:14.290533 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df34b788-d36b-4e5c-8f52-1e4809cda474 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e9ed4d47aef4140bfb2d3bb798cc559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:20:14.291172 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df34b788-d36b-4e5c-8f52-1e4809cda474 service neutron] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:20:14.291452 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df34b788-d36b-4e5c-8f52-1e4809cda474 service neutron] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:20:14.291722 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df34b788-d36b-4e5c-8f52-1e4809cda474 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:20:14.293307 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-df34b788-d36b-4e5c-8f52-1e4809cda474 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:20:14.399390 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df34b788-d36b-4e5c-8f52-1e4809cda474 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:20:14.400753 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df34b788-d36b-4e5c-8f52-1e4809cda474 service neutron] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:20:14.418533 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 731/1470] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:20:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:20:23.828046 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=22 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:20:24.416348 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=66 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:20:33.870240 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b96ee1-82a0-46c1-b71b-cba76f7ebc51 None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:20:33.870793 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b96ee1-82a0-46c1-b71b-cba76f7ebc51 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:20:33.870793 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b96ee1-82a0-46c1-b71b-cba76f7ebc51 None None] PATH_INFO: `/` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:20:33.872199 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 740/1471] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:20:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:20:33.877473 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee351f11-e32b-470e-ae1c-64ab408b9070 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:20:33.877901 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee351f11-e32b-470e-ae1c-64ab408b9070 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:20:33.878052 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee351f11-e32b-470e-ae1c-64ab408b9070 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:20:33.942995 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-ee351f11-e32b-470e-ae1c-64ab408b9070 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:20:33.944195 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-ee351f11-e32b-470e-ae1c-64ab408b9070 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:20:34.042213 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 732/1472] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:20:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:20:34.343061 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916dbbbd-f554-4ece-b82e-3d54137b388a None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:20:34.343346 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916dbbbd-f554-4ece-b82e-3d54137b388a None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:20:34.343620 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916dbbbd-f554-4ece-b82e-3d54137b388a None None] PATH_INFO: `/` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:20:34.344939 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 741/1473] 10.222.0.18 () {54 vars in 967 bytes} [Wed Aug 24 17:20:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:20:34.349728 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c86653c-0747-4a0e-a513-b6afef34a257 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:20:34.350016 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c86653c-0747-4a0e-a513-b6afef34a257 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:20:34.350291 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c86653c-0747-4a0e-a513-b6afef34a257 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:20:34.425597 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-4c86653c-0747-4a0e-a513-b6afef34a257 None None] MFA Rules not processed for user `3553e522363046f6828fd61b2a0642ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:20:34.426801 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-4c86653c-0747-4a0e-a513-b6afef34a257 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:20:34.447917 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:20:34.503037 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 733/1474] 10.222.0.18 () {58 vars in 1064 bytes} [Wed Aug 24 17:20:34 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:20:34.510485 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916dbbbd-f554-4ece-b82e-3d54137b388a None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:20:34.512029 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-916dbbbd-f554-4ece-b82e-3d54137b388a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:20:34.614377 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916dbbbd-f554-4ece-b82e-3d54137b388a None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:20:34.686328 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a3b432-f471-4eea-9758-0a21d54df508 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3553e522363046f6828fd61b2a0642ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:20:34.686951 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a3b432-f471-4eea-9758-0a21d54df508 service glance] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:20:34.687230 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a3b432-f471-4eea-9758-0a21d54df508 service glance] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:20:34.687497 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a3b432-f471-4eea-9758-0a21d54df508 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:20:34.689081 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-02a3b432-f471-4eea-9758-0a21d54df508 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:20:34.741644 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:20:34.795330 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a3b432-f471-4eea-9758-0a21d54df508 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:20:34.797044 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a3b432-f471-4eea-9758-0a21d54df508 service glance] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:20:34.821568 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 742/1475] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:20:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:20:36.850359 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9943a414-6154-4e4f-ade4-2eb23f2060d9 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:20:36.850976 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9943a414-6154-4e4f-ade4-2eb23f2060d9 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:20:36.850976 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9943a414-6154-4e4f-ade4-2eb23f2060d9 None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:20:36.852316 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 734/1476] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:20:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:20:36.857503 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea03ac6f-2f9a-4d2c-9ccd-8689eb7bc04e None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:20:36.857963 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea03ac6f-2f9a-4d2c-9ccd-8689eb7bc04e None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:20:36.858070 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea03ac6f-2f9a-4d2c-9ccd-8689eb7bc04e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:20:36.923855 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-ea03ac6f-2f9a-4d2c-9ccd-8689eb7bc04e None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:20:36.925043 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-ea03ac6f-2f9a-4d2c-9ccd-8689eb7bc04e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:20:37.020434 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 743/1477] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:20:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:20:37.294646 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9943a414-6154-4e4f-ade4-2eb23f2060d9 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:20:37.296222 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-9943a414-6154-4e4f-ade4-2eb23f2060d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:20:37.427495 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9943a414-6154-4e4f-ade4-2eb23f2060d9 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:20:37.476519 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:20:37.520974 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d2c57e-75e0-40eb-94a9-57cf4c287629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:20:37.521578 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d2c57e-75e0-40eb-94a9-57cf4c287629 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:20:37.521858 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d2c57e-75e0-40eb-94a9-57cf4c287629 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:20:37.522131 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d2c57e-75e0-40eb-94a9-57cf4c287629 admin admin] PATH_INFO: `/v3/projects` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:20:37.523386 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d2c57e-75e0-40eb-94a9-57cf4c287629 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:20:37.525178 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: WARNING py.warnings [None req-a4d2c57e-75e0-40eb-94a9-57cf4c287629 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 17:20:37.525178 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: warnings.warn(msg) Aug 24 17:20:37.525178 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: Aug 24 17:20:37.526317 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d2c57e-75e0-40eb-94a9-57cf4c287629 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:20:37.546770 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 735/1478] 10.222.0.18 () {56 vars in 1108 bytes} [Wed Aug 24 17:20:37 2022] GET /identity/v3/projects => generated 2553 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:20:39.475369 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071fbcf3-f6ec-40cb-a90a-c2346fef6544 None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:20:39.475898 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071fbcf3-f6ec-40cb-a90a-c2346fef6544 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:20:39.475898 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071fbcf3-f6ec-40cb-a90a-c2346fef6544 None None] PATH_INFO: `/` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:20:39.477229 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 744/1479] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:20:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:20:39.482366 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7672a1c2-4cf4-4677-be72-74ca0d6d0b0d None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:20:39.482810 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7672a1c2-4cf4-4677-be72-74ca0d6d0b0d None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:20:39.482940 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7672a1c2-4cf4-4677-be72-74ca0d6d0b0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:20:39.550054 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-7672a1c2-4cf4-4677-be72-74ca0d6d0b0d None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:20:39.551348 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-7672a1c2-4cf4-4677-be72-74ca0d6d0b0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:20:39.646352 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 736/1480] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:20:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:20:39.938850 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071fbcf3-f6ec-40cb-a90a-c2346fef6544 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:20:39.940430 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-071fbcf3-f6ec-40cb-a90a-c2346fef6544 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:20:39.950244 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:20:40.083306 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071fbcf3-f6ec-40cb-a90a-c2346fef6544 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:20:40.166145 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:20:40.197200 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143411b6-cc1c-4ddc-bff5-8c5f0f9c7106 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557aca9255cb4bbb8af52e796c5c5177', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:20:40.197806 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143411b6-cc1c-4ddc-bff5-8c5f0f9c7106 service nova] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:20:40.198083 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143411b6-cc1c-4ddc-bff5-8c5f0f9c7106 service nova] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:20:40.198348 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143411b6-cc1c-4ddc-bff5-8c5f0f9c7106 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:20:40.199973 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-143411b6-cc1c-4ddc-bff5-8c5f0f9c7106 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:20:40.288172 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143411b6-cc1c-4ddc-bff5-8c5f0f9c7106 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:20:40.289768 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143411b6-cc1c-4ddc-bff5-8c5f0f9c7106 service nova] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:20:40.307480 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 745/1481] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:20:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:20:42.380200 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085aac59-1e64-4d49-bf96-8cafece78396 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:20:42.380732 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085aac59-1e64-4d49-bf96-8cafece78396 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:20:42.380732 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085aac59-1e64-4d49-bf96-8cafece78396 None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:20:42.382191 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 737/1482] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:20:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:20:42.387486 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e14928b-89ce-4280-8468-a1c730ba2212 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:20:42.388214 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e14928b-89ce-4280-8468-a1c730ba2212 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:20:42.388312 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e14928b-89ce-4280-8468-a1c730ba2212 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:20:42.452598 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-8e14928b-89ce-4280-8468-a1c730ba2212 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:20:42.453812 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-8e14928b-89ce-4280-8468-a1c730ba2212 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:20:42.500028 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:20:42.558506 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 746/1483] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:20:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:20:42.845915 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085aac59-1e64-4d49-bf96-8cafece78396 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:20:42.847485 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-085aac59-1e64-4d49-bf96-8cafece78396 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:20:42.871762 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:20:42.952307 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085aac59-1e64-4d49-bf96-8cafece78396 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:20:43.026501 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5873232-05d8-4d57-9403-cd21a1fb0ade service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e9ed4d47aef4140bfb2d3bb798cc559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:20:43.027139 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5873232-05d8-4d57-9403-cd21a1fb0ade service neutron] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:20:43.027445 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5873232-05d8-4d57-9403-cd21a1fb0ade service neutron] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:20:43.027707 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5873232-05d8-4d57-9403-cd21a1fb0ade service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:20:43.029370 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-a5873232-05d8-4d57-9403-cd21a1fb0ade service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:20:43.102126 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:20:43.136948 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5873232-05d8-4d57-9403-cd21a1fb0ade service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:20:43.138670 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5873232-05d8-4d57-9403-cd21a1fb0ade service neutron] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:20:43.156832 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 738/1484] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:20:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:20:45.173001 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc7499d-3fec-41c6-8ddd-7e4732fcf956 None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:20:45.173513 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc7499d-3fec-41c6-8ddd-7e4732fcf956 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:20:45.173513 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc7499d-3fec-41c6-8ddd-7e4732fcf956 None None] PATH_INFO: `/` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:20:45.174820 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 747/1485] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:20:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:20:45.180241 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a95043-a6ed-4658-9eee-2d90cc35c18a None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:20:45.180685 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a95043-a6ed-4658-9eee-2d90cc35c18a None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:20:45.180792 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a95043-a6ed-4658-9eee-2d90cc35c18a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:20:45.247336 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-c2a95043-a6ed-4658-9eee-2d90cc35c18a None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:20:45.248553 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-c2a95043-a6ed-4658-9eee-2d90cc35c18a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:20:45.343065 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 739/1486] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:20:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:20:45.636209 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc7499d-3fec-41c6-8ddd-7e4732fcf956 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:20:45.638002 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-abc7499d-3fec-41c6-8ddd-7e4732fcf956 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:20:45.740439 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc7499d-3fec-41c6-8ddd-7e4732fcf956 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:20:45.816272 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:20:45.817915 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2327d6-fa5a-4504-b7a3-42dc15d2f4b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e9ed4d47aef4140bfb2d3bb798cc559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:20:45.818550 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2327d6-fa5a-4504-b7a3-42dc15d2f4b7 service neutron] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:20:45.818823 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2327d6-fa5a-4504-b7a3-42dc15d2f4b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:20:45.819631 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2327d6-fa5a-4504-b7a3-42dc15d2f4b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:20:45.821211 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-9a2327d6-fa5a-4504-b7a3-42dc15d2f4b7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:20:45.927353 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2327d6-fa5a-4504-b7a3-42dc15d2f4b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:20:45.929361 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2327d6-fa5a-4504-b7a3-42dc15d2f4b7 service neutron] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:20:45.947523 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 748/1487] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:20:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:20:47.956158 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547d8cd4-8991-407a-9b10-96f162af6212 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:20:47.956759 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547d8cd4-8991-407a-9b10-96f162af6212 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:20:47.956759 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547d8cd4-8991-407a-9b10-96f162af6212 None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:20:47.958058 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 740/1488] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:20:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:20:47.963948 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd734e67-7b6e-4e0a-978e-64e8bf6f68f9 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:20:47.964417 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd734e67-7b6e-4e0a-978e-64e8bf6f68f9 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:20:47.964527 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd734e67-7b6e-4e0a-978e-64e8bf6f68f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:20:48.029891 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-cd734e67-7b6e-4e0a-978e-64e8bf6f68f9 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:20:48.031108 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-cd734e67-7b6e-4e0a-978e-64e8bf6f68f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:20:48.081876 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:20:48.125565 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 749/1489] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:20:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:20:48.412260 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547d8cd4-8991-407a-9b10-96f162af6212 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:20:48.413925 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-547d8cd4-8991-407a-9b10-96f162af6212 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:20:48.450074 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:20:48.518588 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547d8cd4-8991-407a-9b10-96f162af6212 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:20:48.592465 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064d88de-23ba-497b-81fe-93d964de3324 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e9ed4d47aef4140bfb2d3bb798cc559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:20:48.593074 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064d88de-23ba-497b-81fe-93d964de3324 service neutron] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:20:48.593352 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064d88de-23ba-497b-81fe-93d964de3324 service neutron] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:20:48.593626 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064d88de-23ba-497b-81fe-93d964de3324 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:20:48.595241 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-064d88de-23ba-497b-81fe-93d964de3324 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:20:48.674777 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:20:48.703379 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064d88de-23ba-497b-81fe-93d964de3324 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:20:48.704829 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064d88de-23ba-497b-81fe-93d964de3324 service neutron] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:20:48.723583 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 741/1490] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:20:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:20:51.039057 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9787a2e-9b18-428c-bd8f-559b02e4cf9a None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:20:51.039627 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9787a2e-9b18-428c-bd8f-559b02e4cf9a None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:20:51.039627 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9787a2e-9b18-428c-bd8f-559b02e4cf9a None None] PATH_INFO: `/` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:20:51.040969 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 750/1491] 10.222.0.18 () {54 vars in 927 bytes} [Wed Aug 24 17:20:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:20:51.046345 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed079ba4-6a57-472b-8e4d-6fd5e54abcd6 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:20:51.046762 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed079ba4-6a57-472b-8e4d-6fd5e54abcd6 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:20:51.046872 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed079ba4-6a57-472b-8e4d-6fd5e54abcd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:20:51.112000 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-ed079ba4-6a57-472b-8e4d-6fd5e54abcd6 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:20:51.113208 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-ed079ba4-6a57-472b-8e4d-6fd5e54abcd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:20:51.208561 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 742/1492] 10.222.0.18 () {58 vars in 1024 bytes} [Wed Aug 24 17:20:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:20:51.495803 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9787a2e-9b18-428c-bd8f-559b02e4cf9a None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:20:51.497326 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-d9787a2e-9b18-428c-bd8f-559b02e4cf9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:20:51.602110 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9787a2e-9b18-428c-bd8f-559b02e4cf9a None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:20:51.673381 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4200d123-eed0-4a31-86e0-a82e37c0884d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e9ed4d47aef4140bfb2d3bb798cc559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:20:51.673984 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4200d123-eed0-4a31-86e0-a82e37c0884d service neutron] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:20:51.674260 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4200d123-eed0-4a31-86e0-a82e37c0884d service neutron] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:20:51.674527 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4200d123-eed0-4a31-86e0-a82e37c0884d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:20:51.676140 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-4200d123-eed0-4a31-86e0-a82e37c0884d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:20:51.685364 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:20:51.785023 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4200d123-eed0-4a31-86e0-a82e37c0884d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:20:51.786396 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4200d123-eed0-4a31-86e0-a82e37c0884d service neutron] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:20:51.804119 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 751/1493] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:20:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:21:01.206800 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=82 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:21:01.801923 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=50 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:22:10.719397 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f580bdea-0f63-4ac5-bc1a-d4d04204ea9f None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:22:10.719983 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f580bdea-0f63-4ac5-bc1a-d4d04204ea9f None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:22:10.719983 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f580bdea-0f63-4ac5-bc1a-d4d04204ea9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:22:10.794946 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-f580bdea-0f63-4ac5-bc1a-d4d04204ea9f None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:22:10.796181 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-f580bdea-0f63-4ac5-bc1a-d4d04204ea9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:22:10.893624 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 743/1494] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:22:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:22:10.901677 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ac54db-d4cd-4bbb-bea1-e8a97db9eee2 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:22:10.902186 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ac54db-d4cd-4bbb-bea1-e8a97db9eee2 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:22:10.902294 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ac54db-d4cd-4bbb-bea1-e8a97db9eee2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:22:10.976206 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-a8ac54db-d4cd-4bbb-bea1-e8a97db9eee2 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:22:10.977408 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-a8ac54db-d4cd-4bbb-bea1-e8a97db9eee2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:22:11.073777 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 752/1495] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:22:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:22:11.078465 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f580bdea-0f63-4ac5-bc1a-d4d04204ea9f None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:22:11.079724 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-f580bdea-0f63-4ac5-bc1a-d4d04204ea9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:22:11.091047 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f580bdea-0f63-4ac5-bc1a-d4d04204ea9f None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:22:11.153615 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:22:11.187533 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f108b724-adab-4ba3-963f-edd11e94c767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:22:11.188137 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f108b724-adab-4ba3-963f-edd11e94c767 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:22:11.188414 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f108b724-adab-4ba3-963f-edd11e94c767 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:22:11.188686 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f108b724-adab-4ba3-963f-edd11e94c767 admin admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:22:11.189329 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f108b724-adab-4ba3-963f-edd11e94c767 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:22:11.189629 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f108b724-adab-4ba3-963f-edd11e94c767 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:22:11.190634 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: WARNING py.warnings [None req-f108b724-adab-4ba3-963f-edd11e94c767 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 17:22:11.190634 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: warnings.warn(msg) Aug 24 17:22:11.190634 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: Aug 24 17:22:11.191538 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f108b724-adab-4ba3-963f-edd11e94c767 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:22:11.203064 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 744/1496] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 24 17:22:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:22:11.211260 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ac54db-d4cd-4bbb-bea1-e8a97db9eee2 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:22:11.220280 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ac54db-d4cd-4bbb-bea1-e8a97db9eee2 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:22:11.283784 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:22:11.317290 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c046a9-e071-438a-a15b-c895aff95b58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:22:11.317960 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c046a9-e071-438a-a15b-c895aff95b58 admin admin] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:22:11.318250 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c046a9-e071-438a-a15b-c895aff95b58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:22:11.318518 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c046a9-e071-438a-a15b-c895aff95b58 admin admin] PATH_INFO: `/v3/projects` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:22:11.319264 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c046a9-e071-438a-a15b-c895aff95b58 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:22:11.321038 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: WARNING py.warnings [None req-85c046a9-e071-438a-a15b-c895aff95b58 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 17:22:11.321038 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: warnings.warn(msg) Aug 24 17:22:11.321038 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: Aug 24 17:22:11.322766 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c046a9-e071-438a-a15b-c895aff95b58 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:22:11.569474 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 753/1497] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 24 17:22:11 2022] POST /identity/v3/projects => generated 379 bytes in 359 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:22:11.574223 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f108b724-adab-4ba3-963f-edd11e94c767 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:22:11.575763 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-f108b724-adab-4ba3-963f-edd11e94c767 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:22:11.608522 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f108b724-adab-4ba3-963f-edd11e94c767 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:22:11.702789 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f238dea-b4b0-4a8d-92b4-74b39b88f091 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:22:11.703422 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f238dea-b4b0-4a8d-92b4-74b39b88f091 admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:22:11.703706 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f238dea-b4b0-4a8d-92b4-74b39b88f091 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:22:11.703980 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f238dea-b4b0-4a8d-92b4-74b39b88f091 admin admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:22:11.704689 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f238dea-b4b0-4a8d-92b4-74b39b88f091 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:22:11.705708 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: WARNING py.warnings [None req-0f238dea-b4b0-4a8d-92b4-74b39b88f091 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 17:22:11.705708 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: warnings.warn(msg) Aug 24 17:22:11.705708 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: Aug 24 17:22:11.706935 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f238dea-b4b0-4a8d-92b4-74b39b88f091 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:22:11.814042 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 745/1498] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 24 17:22:11 2022] POST /identity/v3/users => generated 290 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:22:11.818103 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c046a9-e071-438a-a15b-c895aff95b58 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:22:11.819655 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-85c046a9-e071-438a-a15b-c895aff95b58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:22:11.880940 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c046a9-e071-438a-a15b-c895aff95b58 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:22:11.961723 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:22:11.972533 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5bef2e-880a-4389-bff2-8d55ac9f8e5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:22:11.976068 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5bef2e-880a-4389-bff2-8d55ac9f8e5d admin admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:22:11.976068 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5bef2e-880a-4389-bff2-8d55ac9f8e5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:22:11.976068 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5bef2e-880a-4389-bff2-8d55ac9f8e5d admin admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:22:11.976068 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5bef2e-880a-4389-bff2-8d55ac9f8e5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:22:11.976068 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: WARNING py.warnings [None req-0a5bef2e-880a-4389-bff2-8d55ac9f8e5d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 17:22:11.976068 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: warnings.warn(msg) Aug 24 17:22:11.976068 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: Aug 24 17:22:11.976679 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5bef2e-880a-4389-bff2-8d55ac9f8e5d admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:22:11.989049 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 754/1499] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:22:11 2022] GET /identity/v3/roles => generated 1420 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:22:11.993092 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f238dea-b4b0-4a8d-92b4-74b39b88f091 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:22:11.995499 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-0f238dea-b4b0-4a8d-92b4-74b39b88f091 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:22:12.011326 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f238dea-b4b0-4a8d-92b4-74b39b88f091 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:22:12.037513 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:22:12.108919 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cb4100-b1f5-4226-9bcf-950611baf271 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:22:12.112534 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cb4100-b1f5-4226-9bcf-950611baf271 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:22:12.112534 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cb4100-b1f5-4226-9bcf-950611baf271 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:22:12.112534 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cb4100-b1f5-4226-9bcf-950611baf271 admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:22:12.112534 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cb4100-b1f5-4226-9bcf-950611baf271 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:22:12.112534 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: WARNING py.warnings [None req-d1cb4100-b1f5-4226-9bcf-950611baf271 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 17:22:12.112534 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: warnings.warn(msg) Aug 24 17:22:12.112534 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: Aug 24 17:22:12.113166 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cb4100-b1f5-4226-9bcf-950611baf271 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:22:12.128828 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 746/1500] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:22:11 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:22:12.133239 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5bef2e-880a-4389-bff2-8d55ac9f8e5d admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:22:12.134780 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-0a5bef2e-880a-4389-bff2-8d55ac9f8e5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:22:12.182295 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5bef2e-880a-4389-bff2-8d55ac9f8e5d admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:22:12.276095 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d84a1e8-fbdb-4549-850e-eb0bcb052ea8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:22:12.276750 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d84a1e8-fbdb-4549-850e-eb0bcb052ea8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:22:12.277029 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d84a1e8-fbdb-4549-850e-eb0bcb052ea8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:22:12.277303 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d84a1e8-fbdb-4549-850e-eb0bcb052ea8 admin admin] PATH_INFO: `/v3/projects/04b3e012936c4748acde68cc43755a5c/users/847df36bc9ca4b5ca15a184a81a562d9/roles/60dc7e16aee746d5a3e8aa27d7d8525c` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:22:12.299791 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d84a1e8-fbdb-4549-850e-eb0bcb052ea8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=04b3e012936c4748acde68cc43755a5c, user_id=847df36bc9ca4b5ca15a184a81a562d9, role_id=60dc7e16aee746d5a3e8aa27d7d8525c)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:22:12.300802 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: WARNING py.warnings [None req-0d84a1e8-fbdb-4549-850e-eb0bcb052ea8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 17:22:12.300802 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: warnings.warn(msg) Aug 24 17:22:12.300802 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: Aug 24 17:22:12.302711 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d84a1e8-fbdb-4549-850e-eb0bcb052ea8 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:22:12.382772 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 755/1501] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 24 17:22:12 2022] PUT /identity/v3/projects/04b3e012936c4748acde68cc43755a5c/users/847df36bc9ca4b5ca15a184a81a562d9/roles/60dc7e16aee746d5a3e8aa27d7d8525c => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:22:12.395682 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae893c93-e838-474a-87e5-2a58dbfe5f85 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:22:12.398762 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae893c93-e838-474a-87e5-2a58dbfe5f85 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:22:12.398762 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae893c93-e838-474a-87e5-2a58dbfe5f85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:22:12.456460 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-ae893c93-e838-474a-87e5-2a58dbfe5f85 None None] MFA Rules not processed for user `847df36bc9ca4b5ca15a184a81a562d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:22:12.457724 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-ae893c93-e838-474a-87e5-2a58dbfe5f85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:22:12.536822 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 747/1502] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:22:12 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:22:12.550557 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aeb05b-9e9b-4772-9fd4-3dcedeb5a8ac None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:22:12.550879 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aeb05b-9e9b-4772-9fd4-3dcedeb5a8ac None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:22:12.551184 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aeb05b-9e9b-4772-9fd4-3dcedeb5a8ac None None] PATH_INFO: `/` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:22:12.552518 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 756/1503] 10.222.0.18 () {54 vars in 964 bytes} [Wed Aug 24 17:22:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:22:12.557438 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b685cd6-1722-40ab-97a2-23b9aa81d5ad None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:22:12.557726 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b685cd6-1722-40ab-97a2-23b9aa81d5ad None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:22:12.558000 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b685cd6-1722-40ab-97a2-23b9aa81d5ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:22:12.589160 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:22:12.632002 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-2b685cd6-1722-40ab-97a2-23b9aa81d5ad None None] MFA Rules not processed for user `be06246d47584b529150ceb98a5cfd95`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:22:12.633247 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-2b685cd6-1722-40ab-97a2-23b9aa81d5ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:22:12.708021 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 748/1504] 10.222.0.18 () {58 vars in 1061 bytes} [Wed Aug 24 17:22:12 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:22:12.716358 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66aeb05b-9e9b-4772-9fd4-3dcedeb5a8ac None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:22:12.717903 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-66aeb05b-9e9b-4772-9fd4-3dcedeb5a8ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:22:12.767330 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:22:12.821312 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66aeb05b-9e9b-4772-9fd4-3dcedeb5a8ac None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:22:12.893619 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76be48db-de2b-4af2-b9e5-45493ecfe1ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be06246d47584b529150ceb98a5cfd95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:22:12.894147 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76be48db-de2b-4af2-b9e5-45493ecfe1ac service cinder] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:22:12.894431 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76be48db-de2b-4af2-b9e5-45493ecfe1ac service cinder] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:22:12.894704 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76be48db-de2b-4af2-b9e5-45493ecfe1ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:22:12.896334 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-76be48db-de2b-4af2-b9e5-45493ecfe1ac service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:22:12.992253 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76be48db-de2b-4af2-b9e5-45493ecfe1ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:22:12.993681 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76be48db-de2b-4af2-b9e5-45493ecfe1ac service cinder] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:22:12.997231 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:22:13.011614 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 757/1505] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:22:12 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:22:13.056537 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb12eb6-3fbc-436e-8e58-aac9d5d8b983 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:22:13.056949 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb12eb6-3fbc-436e-8e58-aac9d5d8b983 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:22:13.057125 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb12eb6-3fbc-436e-8e58-aac9d5d8b983 None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:22:13.058399 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 749/1506] 10.222.0.18 () {52 vars in 830 bytes} [Wed Aug 24 17:22:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:22:13.061944 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76be48db-de2b-4af2-b9e5-45493ecfe1ac service cinder] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:22:13.063202 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-76be48db-de2b-4af2-b9e5-45493ecfe1ac service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:22:13.159019 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76be48db-de2b-4af2-b9e5-45493ecfe1ac service cinder] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:22:13.251517 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5f584a-b116-45f7-b18d-eb7f55233523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:22:13.252144 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5f584a-b116-45f7-b18d-eb7f55233523 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:22:13.252417 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5f584a-b116-45f7-b18d-eb7f55233523 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:22:13.252683 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5f584a-b116-45f7-b18d-eb7f55233523 admin admin] PATH_INFO: `/v3/users/847df36bc9ca4b5ca15a184a81a562d9` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:22:13.266738 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5f584a-b116-45f7-b18d-eb7f55233523 admin admin] RBAC: Authorizing `identity:delete_user(user_id=847df36bc9ca4b5ca15a184a81a562d9)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:22:13.267801 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: WARNING py.warnings [None req-1c5f584a-b116-45f7-b18d-eb7f55233523 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 17:22:13.267801 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: warnings.warn(msg) Aug 24 17:22:13.267801 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: Aug 24 17:22:13.269031 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5f584a-b116-45f7-b18d-eb7f55233523 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:22:13.278420 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:22:13.425430 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-1c5f584a-b116-45f7-b18d-eb7f55233523 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '847df36bc9ca4b5ca15a184a81a562d9'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:22:13.449765 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-1c5f584a-b116-45f7-b18d-eb7f55233523 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '847df36bc9ca4b5ca15a184a81a562d9'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:22:13.508940 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-1c5f584a-b116-45f7-b18d-eb7f55233523 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '847df36bc9ca4b5ca15a184a81a562d9'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:22:13.509377 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-1c5f584a-b116-45f7-b18d-eb7f55233523 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '847df36bc9ca4b5ca15a184a81a562d9'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:22:13.530923 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-1c5f584a-b116-45f7-b18d-eb7f55233523 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '847df36bc9ca4b5ca15a184a81a562d9'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:22:13.537025 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 758/1507] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 24 17:22:13 2022] DELETE /identity/v3/users/847df36bc9ca4b5ca15a184a81a562d9 => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:22:13.542636 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5f8998-708f-4bee-a502-ab6848d3f242 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:22:13.542636 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5f8998-708f-4bee-a502-ab6848d3f242 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:22:13.542636 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5f8998-708f-4bee-a502-ab6848d3f242 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:22:13.604156 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-ae5f8998-708f-4bee-a502-ab6848d3f242 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:22:13.605905 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-ae5f8998-708f-4bee-a502-ab6848d3f242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:22:13.681709 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:22:13.813360 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 750/1508] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:22:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:22:13.822922 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5f584a-b116-45f7-b18d-eb7f55233523 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:22:13.824139 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-1c5f584a-b116-45f7-b18d-eb7f55233523 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:22:13.921020 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5f584a-b116-45f7-b18d-eb7f55233523 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:22:13.974082 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:22:13.992606 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7b544c-ee40-4404-8bc0-5bcb443b3901 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e9ed4d47aef4140bfb2d3bb798cc559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:22:13.993304 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7b544c-ee40-4404-8bc0-5bcb443b3901 service neutron] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:22:13.993585 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7b544c-ee40-4404-8bc0-5bcb443b3901 service neutron] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:22:13.993851 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7b544c-ee40-4404-8bc0-5bcb443b3901 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:22:13.995485 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-3e7b544c-ee40-4404-8bc0-5bcb443b3901 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:22:14.107742 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7b544c-ee40-4404-8bc0-5bcb443b3901 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:22:14.109168 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7b544c-ee40-4404-8bc0-5bcb443b3901 service neutron] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:22:14.127513 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 759/1509] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:22:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:22:14.556526 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5f8998-708f-4bee-a502-ab6848d3f242 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:22:14.558468 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-ae5f8998-708f-4bee-a502-ab6848d3f242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:22:14.570093 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5f8998-708f-4bee-a502-ab6848d3f242 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:22:14.674730 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4c47c2-1d8d-4eea-b565-a36ae7b625b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:22:14.675389 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4c47c2-1d8d-4eea-b565-a36ae7b625b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:22:14.675669 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4c47c2-1d8d-4eea-b565-a36ae7b625b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:22:14.675937 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4c47c2-1d8d-4eea-b565-a36ae7b625b7 admin admin] PATH_INFO: `/v3/projects/04b3e012936c4748acde68cc43755a5c` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:22:14.686277 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4c47c2-1d8d-4eea-b565-a36ae7b625b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=04b3e012936c4748acde68cc43755a5c)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:22:14.687340 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: WARNING py.warnings [None req-bf4c47c2-1d8d-4eea-b565-a36ae7b625b7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 17:22:14.687340 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: warnings.warn(msg) Aug 24 17:22:14.687340 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: Aug 24 17:22:14.688560 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4c47c2-1d8d-4eea-b565-a36ae7b625b7 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:22:14.805489 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-bf4c47c2-1d8d-4eea-b565-a36ae7b625b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04b3e012936c4748acde68cc43755a5c', 'request_id': 'req-bf4c47c2-1d8d-4eea-b565-a36ae7b625b7'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:22:14.880393 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-bf4c47c2-1d8d-4eea-b565-a36ae7b625b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04b3e012936c4748acde68cc43755a5c', 'request_id': 'req-bf4c47c2-1d8d-4eea-b565-a36ae7b625b7'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:22:14.894358 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-bf4c47c2-1d8d-4eea-b565-a36ae7b625b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04b3e012936c4748acde68cc43755a5c', 'request_id': 'req-bf4c47c2-1d8d-4eea-b565-a36ae7b625b7'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:22:14.918376 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-bf4c47c2-1d8d-4eea-b565-a36ae7b625b7 admin admin] The token cache is being invalidate because project 04b3e012936c4748acde68cc43755a5c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81732) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:22:14.918667 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-bf4c47c2-1d8d-4eea-b565-a36ae7b625b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:22:14.919221 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-bf4c47c2-1d8d-4eea-b565-a36ae7b625b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:22:14.920628 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 751/1510] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 24 17:22:14 2022] DELETE /identity/v3/projects/04b3e012936c4748acde68cc43755a5c => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:22:24.125291 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=62 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:22:24.915733 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:23:44.545941 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72efcad-bd8c-4702-84e1-d09d4747a339 None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:23:44.546815 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72efcad-bd8c-4702-84e1-d09d4747a339 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:23:44.546815 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72efcad-bd8c-4702-84e1-d09d4747a339 None None] PATH_INFO: `/` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:23:44.548556 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 760/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 17:23:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:23:44.555899 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e67e2d-87cb-427d-a35d-303c8ad051b6 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:23:44.556722 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e67e2d-87cb-427d-a35d-303c8ad051b6 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:23:44.556722 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e67e2d-87cb-427d-a35d-303c8ad051b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:23:44.657902 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-a7e67e2d-87cb-427d-a35d-303c8ad051b6 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:23:44.659210 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-a7e67e2d-87cb-427d-a35d-303c8ad051b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:23:44.789695 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 752/1512] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 17:23:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:23:44.948516 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72efcad-bd8c-4702-84e1-d09d4747a339 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:23:44.951064 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-f72efcad-bd8c-4702-84e1-d09d4747a339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:23:45.121631 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72efcad-bd8c-4702-84e1-d09d4747a339 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:23:45.128102 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69664a8f-119f-4a1b-9629-a9c47a6f4061 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557aca9255cb4bbb8af52e796c5c5177', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:23:45.128739 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69664a8f-119f-4a1b-9629-a9c47a6f4061 service nova] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:23:45.129021 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69664a8f-119f-4a1b-9629-a9c47a6f4061 service nova] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:23:45.129297 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69664a8f-119f-4a1b-9629-a9c47a6f4061 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:23:45.139474 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69664a8f-119f-4a1b-9629-a9c47a6f4061 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:23:45.141730 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69664a8f-119f-4a1b-9629-a9c47a6f4061 service nova] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:23:45.153016 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 761/1513] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:23:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:23:46.030744 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac06d02-4b8f-4c61-aea3-439c8e8995ea None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:23:46.031341 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac06d02-4b8f-4c61-aea3-439c8e8995ea None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:23:46.031499 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac06d02-4b8f-4c61-aea3-439c8e8995ea None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:23:46.033138 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 753/1514] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 17:23:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:23:46.038325 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fff5467-dd67-462e-bcb2-d0a13a445ac9 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:23:46.038850 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fff5467-dd67-462e-bcb2-d0a13a445ac9 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:23:46.038850 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fff5467-dd67-462e-bcb2-d0a13a445ac9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:23:46.061819 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-8fff5467-dd67-462e-bcb2-d0a13a445ac9 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:23:46.063127 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-8fff5467-dd67-462e-bcb2-d0a13a445ac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:23:46.071520 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 762/1515] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 17:23:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:23:46.230415 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac06d02-4b8f-4c61-aea3-439c8e8995ea None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:23:46.249517 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac06d02-4b8f-4c61-aea3-439c8e8995ea None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:23:46.258264 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5c6c06-2d4d-4375-bf7f-90018a3b526b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557aca9255cb4bbb8af52e796c5c5177', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:23:46.259169 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5c6c06-2d4d-4375-bf7f-90018a3b526b service nova] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:23:46.259490 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5c6c06-2d4d-4375-bf7f-90018a3b526b service nova] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:23:46.259773 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5c6c06-2d4d-4375-bf7f-90018a3b526b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:23:46.274952 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5c6c06-2d4d-4375-bf7f-90018a3b526b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:23:46.277735 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5c6c06-2d4d-4375-bf7f-90018a3b526b service nova] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:23:46.292852 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 754/1516] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:23:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:23:47.580826 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7be2b3-ef52-4ba5-9f34-8abc1e53676f None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:23:47.581600 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7be2b3-ef52-4ba5-9f34-8abc1e53676f None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:23:47.581600 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7be2b3-ef52-4ba5-9f34-8abc1e53676f None None] PATH_INFO: `/v3` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:23:47.583497 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 763/1517] 10.222.0.50 () {54 vars in 925 bytes} [Wed Aug 24 17:23:47 2022] GET /identity/v3 => generated 255 bytes in 5 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:23:47.592149 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ce2c42-e3bd-4a4c-86a6-50662d2ed7ed None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:23:47.592748 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ce2c42-e3bd-4a4c-86a6-50662d2ed7ed None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:23:47.592748 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ce2c42-e3bd-4a4c-86a6-50662d2ed7ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:23:47.667520 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-c9ce2c42-e3bd-4a4c-86a6-50662d2ed7ed None None] MFA Rules not processed for user `03fc77c6107e430f90f7b6f7cfd4ec02`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:23:47.668786 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-c9ce2c42-e3bd-4a4c-86a6-50662d2ed7ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:23:47.688248 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:23:47.759838 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 755/1518] 10.222.0.50 () {58 vars in 1015 bytes} [Wed Aug 24 17:23:47 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:23:56.055934 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=56 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:23:57.175566 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee042db1-a095-46b6-b866-10f94b540ab1 None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:23:57.176208 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee042db1-a095-46b6-b866-10f94b540ab1 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:23:57.176208 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee042db1-a095-46b6-b866-10f94b540ab1 None None] PATH_INFO: `/` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:23:57.177361 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 764/1519] 10.222.0.18 () {54 vars in 961 bytes} [Wed Aug 24 17:23:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:23:57.182451 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1ede67-2e4d-4038-a09c-c1975b3f0227 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:23:57.182938 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1ede67-2e4d-4038-a09c-c1975b3f0227 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:23:57.183056 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1ede67-2e4d-4038-a09c-c1975b3f0227 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:23:57.202130 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-5b1ede67-2e4d-4038-a09c-c1975b3f0227 None None] MFA Rules not processed for user `03fc77c6107e430f90f7b6f7cfd4ec02`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:23:57.203380 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-5b1ede67-2e4d-4038-a09c-c1975b3f0227 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:23:57.211840 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 756/1520] 10.222.0.18 () {58 vars in 1058 bytes} [Wed Aug 24 17:23:57 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:23:57.219728 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee042db1-a095-46b6-b866-10f94b540ab1 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:23:57.234078 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee042db1-a095-46b6-b866-10f94b540ab1 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:23:57.239174 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f310b6f3-6828-432a-8b56-844d00104724 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03fc77c6107e430f90f7b6f7cfd4ec02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:23:57.239784 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f310b6f3-6828-432a-8b56-844d00104724 service placement] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:23:57.240067 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f310b6f3-6828-432a-8b56-844d00104724 service placement] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:23:57.240338 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f310b6f3-6828-432a-8b56-844d00104724 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:23:57.248883 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f310b6f3-6828-432a-8b56-844d00104724 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:23:57.250446 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f310b6f3-6828-432a-8b56-844d00104724 service placement] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:23:57.260173 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 765/1521] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:23:57 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:23:57.567950 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b345477e-09fd-4204-8887-a312fb0a0e2b None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:23:57.568232 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b345477e-09fd-4204-8887-a312fb0a0e2b None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:23:57.568505 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b345477e-09fd-4204-8887-a312fb0a0e2b None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:23:57.569899 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 757/1522] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 17:23:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:23:57.575559 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06da67f-4c70-4e63-bafc-06473353d2e2 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:23:57.575840 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06da67f-4c70-4e63-bafc-06473353d2e2 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:23:57.576105 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06da67f-4c70-4e63-bafc-06473353d2e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:23:57.595599 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-e06da67f-4c70-4e63-bafc-06473353d2e2 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:23:57.602265 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-e06da67f-4c70-4e63-bafc-06473353d2e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:23:57.604526 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 766/1523] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 17:23:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:23:57.758769 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b345477e-09fd-4204-8887-a312fb0a0e2b None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:23:57.778796 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b345477e-09fd-4204-8887-a312fb0a0e2b None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:23:57.785439 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09e6b9e-150d-42d1-a810-d2b256234dda service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557aca9255cb4bbb8af52e796c5c5177', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:23:57.786214 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09e6b9e-150d-42d1-a810-d2b256234dda service nova] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:23:57.787170 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09e6b9e-150d-42d1-a810-d2b256234dda service nova] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:23:57.787363 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09e6b9e-150d-42d1-a810-d2b256234dda service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:23:57.803147 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09e6b9e-150d-42d1-a810-d2b256234dda service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:23:57.803147 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09e6b9e-150d-42d1-a810-d2b256234dda service nova] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:23:57.812704 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 758/1524] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:23:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:23:58.642613 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3de947-75aa-40ec-944f-fe077417e101 None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:23:58.643269 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3de947-75aa-40ec-944f-fe077417e101 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:23:58.643269 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3de947-75aa-40ec-944f-fe077417e101 None None] PATH_INFO: `/` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:23:58.644545 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 767/1525] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 17:23:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:23:58.649225 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5bf011-8700-49d7-a106-58a959442a76 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:23:58.649633 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5bf011-8700-49d7-a106-58a959442a76 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:23:58.649772 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5bf011-8700-49d7-a106-58a959442a76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:23:58.670033 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-7d5bf011-8700-49d7-a106-58a959442a76 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:23:58.671316 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-7d5bf011-8700-49d7-a106-58a959442a76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:23:58.679267 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 759/1526] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 17:23:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:23:58.829329 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3de947-75aa-40ec-944f-fe077417e101 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:23:58.844074 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3de947-75aa-40ec-944f-fe077417e101 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:23:58.849111 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d5486d-8bdd-4ca3-9983-8de8d7a2a3b1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557aca9255cb4bbb8af52e796c5c5177', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:23:58.849706 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d5486d-8bdd-4ca3-9983-8de8d7a2a3b1 service nova] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:23:58.849981 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d5486d-8bdd-4ca3-9983-8de8d7a2a3b1 service nova] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:23:58.850248 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d5486d-8bdd-4ca3-9983-8de8d7a2a3b1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:23:58.859421 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d5486d-8bdd-4ca3-9983-8de8d7a2a3b1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:23:58.861077 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d5486d-8bdd-4ca3-9983-8de8d7a2a3b1 service nova] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:23:58.870514 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 768/1527] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:23:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:24:02.827556 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a993de-636f-4594-b859-a98459fe9bdf None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:24:02.828120 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a993de-636f-4594-b859-a98459fe9bdf None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:24:02.828120 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a993de-636f-4594-b859-a98459fe9bdf None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:24:02.829456 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 760/1528] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 17:24:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:24:02.834738 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e8ee71-ffbc-4f59-a30f-6d33b3baefb6 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:24:02.835180 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e8ee71-ffbc-4f59-a30f-6d33b3baefb6 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:24:02.835341 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e8ee71-ffbc-4f59-a30f-6d33b3baefb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:24:02.854395 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-70e8ee71-ffbc-4f59-a30f-6d33b3baefb6 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:24:02.856176 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-70e8ee71-ffbc-4f59-a30f-6d33b3baefb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:24:02.864186 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 769/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 17:24:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:24:03.156975 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a993de-636f-4594-b859-a98459fe9bdf None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:24:03.158675 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-d0a993de-636f-4594-b859-a98459fe9bdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:24:03.235698 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a993de-636f-4594-b859-a98459fe9bdf None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:24:03.239278 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1327e1b8-2696-436e-9bcd-5c3849e6eba7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e9ed4d47aef4140bfb2d3bb798cc559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:24:03.239877 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1327e1b8-2696-436e-9bcd-5c3849e6eba7 service neutron] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:24:03.240158 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1327e1b8-2696-436e-9bcd-5c3849e6eba7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:24:03.240428 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1327e1b8-2696-436e-9bcd-5c3849e6eba7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:24:03.249278 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1327e1b8-2696-436e-9bcd-5c3849e6eba7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:24:03.250697 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1327e1b8-2696-436e-9bcd-5c3849e6eba7 service neutron] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:24:03.260131 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 761/1530] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:24:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:24:05.176994 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f10546f-9fd4-4e59-83be-39771575bec4 None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:24:05.177565 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f10546f-9fd4-4e59-83be-39771575bec4 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:24:05.177565 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f10546f-9fd4-4e59-83be-39771575bec4 None None] PATH_INFO: `/` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:24:05.178896 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 770/1531] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 17:24:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:24:05.184822 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0960d829-bd8f-44be-b8cf-68540ef4450f None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:24:05.185276 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0960d829-bd8f-44be-b8cf-68540ef4450f None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:24:05.185389 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0960d829-bd8f-44be-b8cf-68540ef4450f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:24:05.204510 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-0960d829-bd8f-44be-b8cf-68540ef4450f None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:24:05.205769 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-0960d829-bd8f-44be-b8cf-68540ef4450f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:24:05.213757 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 762/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 17:24:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:24:05.499061 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f10546f-9fd4-4e59-83be-39771575bec4 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:24:05.509849 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f10546f-9fd4-4e59-83be-39771575bec4 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:24:05.513701 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f0258b-db0a-45cc-9c62-048dd658f306 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e9ed4d47aef4140bfb2d3bb798cc559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:24:05.514455 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f0258b-db0a-45cc-9c62-048dd658f306 service neutron] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:24:05.515033 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f0258b-db0a-45cc-9c62-048dd658f306 service neutron] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:24:05.515311 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f0258b-db0a-45cc-9c62-048dd658f306 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:24:05.524229 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f0258b-db0a-45cc-9c62-048dd658f306 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:24:05.525661 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f0258b-db0a-45cc-9c62-048dd658f306 service neutron] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:24:05.536025 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 771/1533] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:24:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:24:08.942816 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137ccfce-2432-49c5-9774-9d33ca1c1e21 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:24:08.943490 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137ccfce-2432-49c5-9774-9d33ca1c1e21 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:24:08.943490 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137ccfce-2432-49c5-9774-9d33ca1c1e21 None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:24:08.944777 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 763/1534] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 17:24:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:24:08.952518 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79930c75-5e5f-481d-9346-1a4bd0ac0ab1 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:24:08.953008 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79930c75-5e5f-481d-9346-1a4bd0ac0ab1 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:24:08.953508 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79930c75-5e5f-481d-9346-1a4bd0ac0ab1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:24:08.975113 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-79930c75-5e5f-481d-9346-1a4bd0ac0ab1 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:24:08.976785 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-79930c75-5e5f-481d-9346-1a4bd0ac0ab1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:24:08.984483 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 772/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 17:24:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:24:09.541384 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137ccfce-2432-49c5-9774-9d33ca1c1e21 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:24:09.545638 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-137ccfce-2432-49c5-9774-9d33ca1c1e21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:24:09.610001 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:24:09.622992 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137ccfce-2432-49c5-9774-9d33ca1c1e21 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:24:09.626645 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73a2cbe-973b-4d1b-b264-e63cbcc4d3b1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be06246d47584b529150ceb98a5cfd95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:24:09.627287 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73a2cbe-973b-4d1b-b264-e63cbcc4d3b1 service cinder] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:24:09.627565 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73a2cbe-973b-4d1b-b264-e63cbcc4d3b1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:24:09.627843 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73a2cbe-973b-4d1b-b264-e63cbcc4d3b1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:24:09.636652 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73a2cbe-973b-4d1b-b264-e63cbcc4d3b1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:24:09.638108 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73a2cbe-973b-4d1b-b264-e63cbcc4d3b1 service cinder] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:24:09.648285 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 764/1536] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:24:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:24:11.590664 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7916ca8-51e6-449e-8a43-520632a96019 None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:24:11.591200 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7916ca8-51e6-449e-8a43-520632a96019 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:24:11.591200 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7916ca8-51e6-449e-8a43-520632a96019 None None] PATH_INFO: `/` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:24:11.592522 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 773/1537] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 17:24:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:24:11.597829 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc29129-f15e-4f15-a9d2-b75e194b0110 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:24:11.598256 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc29129-f15e-4f15-a9d2-b75e194b0110 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:24:11.598384 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc29129-f15e-4f15-a9d2-b75e194b0110 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:24:11.617450 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-3dc29129-f15e-4f15-a9d2-b75e194b0110 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:24:11.618647 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-3dc29129-f15e-4f15-a9d2-b75e194b0110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:24:11.627088 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 765/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 17:24:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:24:12.185252 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7916ca8-51e6-449e-8a43-520632a96019 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:24:12.186931 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-d7916ca8-51e6-449e-8a43-520632a96019 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:24:12.199668 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7916ca8-51e6-449e-8a43-520632a96019 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:24:12.203197 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4189a8a-127f-4014-b94b-84863fa01bf8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be06246d47584b529150ceb98a5cfd95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:24:12.203789 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4189a8a-127f-4014-b94b-84863fa01bf8 service cinder] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:24:12.204073 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4189a8a-127f-4014-b94b-84863fa01bf8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:24:12.204347 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4189a8a-127f-4014-b94b-84863fa01bf8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:24:12.213165 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4189a8a-127f-4014-b94b-84863fa01bf8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:24:12.214579 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4189a8a-127f-4014-b94b-84863fa01bf8 service cinder] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:24:12.224618 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 774/1539] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:24:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:24:21.612710 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=11 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:24:22.220336 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=33 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:25:37.128539 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f490ebb-ee46-461a-acc3-4869fd36b103 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:37.131750 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f490ebb-ee46-461a-acc3-4869fd36b103 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:37.131750 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f490ebb-ee46-461a-acc3-4869fd36b103 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:37.157602 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-8f490ebb-ee46-461a-acc3-4869fd36b103 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:25:37.158856 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-8f490ebb-ee46-461a-acc3-4869fd36b103 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:25:37.176837 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 766/1540] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:25:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:25:37.186347 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8688beb-1357-4bae-8afe-4b8cbc956917 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:37.186972 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8688beb-1357-4bae-8afe-4b8cbc956917 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:37.186972 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8688beb-1357-4bae-8afe-4b8cbc956917 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:37.215573 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-e8688beb-1357-4bae-8afe-4b8cbc956917 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:25:37.216786 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-e8688beb-1357-4bae-8afe-4b8cbc956917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:25:37.232500 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 775/1541] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:25:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:25:37.237521 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f490ebb-ee46-461a-acc3-4869fd36b103 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:25:37.248842 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f490ebb-ee46-461a-acc3-4869fd36b103 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:25:37.258652 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cd3c98-cace-4201-bd1f-15227894a4cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:25:37.259670 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cd3c98-cace-4201-bd1f-15227894a4cb admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:37.260175 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cd3c98-cace-4201-bd1f-15227894a4cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:37.260677 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cd3c98-cace-4201-bd1f-15227894a4cb admin admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:37.261645 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cd3c98-cace-4201-bd1f-15227894a4cb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:25:37.262158 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cd3c98-cace-4201-bd1f-15227894a4cb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:25:37.264651 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cd3c98-cace-4201-bd1f-15227894a4cb admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:25:37.282242 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 767/1542] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 24 17:25:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:25:37.290932 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8688beb-1357-4bae-8afe-4b8cbc956917 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:25:37.308865 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8688beb-1357-4bae-8afe-4b8cbc956917 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:25:37.320378 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0d5599-93dd-4969-a53f-4b1418bebf05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:25:37.321033 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0d5599-93dd-4969-a53f-4b1418bebf05 admin admin] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:37.321317 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0d5599-93dd-4969-a53f-4b1418bebf05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:37.321593 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0d5599-93dd-4969-a53f-4b1418bebf05 admin admin] PATH_INFO: `/v3/projects` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:37.322254 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0d5599-93dd-4969-a53f-4b1418bebf05 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:25:37.324337 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0d5599-93dd-4969-a53f-4b1418bebf05 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:25:37.445787 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 776/1543] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 24 17:25:37 2022] POST /identity/v3/projects => generated 377 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:25:37.451188 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cd3c98-cace-4201-bd1f-15227894a4cb admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:25:37.464330 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cd3c98-cace-4201-bd1f-15227894a4cb admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:25:37.476219 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1000ea3-1bc0-47fb-bdde-6047024cf9c2 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:37.476521 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1000ea3-1bc0-47fb-bdde-6047024cf9c2 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:37.476896 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1000ea3-1bc0-47fb-bdde-6047024cf9c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:37.516069 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-a1000ea3-1bc0-47fb-bdde-6047024cf9c2 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:25:37.517354 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-a1000ea3-1bc0-47fb-bdde-6047024cf9c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:25:37.544528 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43c2356-1bfb-4624-8923-b915a54f2501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:25:37.545774 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43c2356-1bfb-4624-8923-b915a54f2501 admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:37.546278 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43c2356-1bfb-4624-8923-b915a54f2501 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:37.546773 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43c2356-1bfb-4624-8923-b915a54f2501 admin admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:37.547730 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43c2356-1bfb-4624-8923-b915a54f2501 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:25:37.550124 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43c2356-1bfb-4624-8923-b915a54f2501 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:25:37.655084 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 777/1544] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:25:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:25:37.656894 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1875099e-9e0c-4863-a9dd-6d609b247964 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:37.657455 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1875099e-9e0c-4863-a9dd-6d609b247964 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:37.657900 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1875099e-9e0c-4863-a9dd-6d609b247964 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:37.667666 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 768/1545] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 24 17:25:37 2022] POST /identity/v3/users => generated 289 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:25:37.669827 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5541abee-d597-4a2f-905e-06c949261b11 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:37.670550 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5541abee-d597-4a2f-905e-06c949261b11 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:37.670826 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5541abee-d597-4a2f-905e-06c949261b11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:37.701972 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-5541abee-d597-4a2f-905e-06c949261b11 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:25:37.702771 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-1875099e-9e0c-4863-a9dd-6d609b247964 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:25:37.703356 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-5541abee-d597-4a2f-905e-06c949261b11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:25:37.704223 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-1875099e-9e0c-4863-a9dd-6d609b247964 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:25:37.712122 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 769/1546] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:25:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:25:37.712661 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 778/1547] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:25:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:25:37.713541 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f96eccd-edb1-4c9e-b183-0f2ae9f32781 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:37.713877 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f96eccd-edb1-4c9e-b183-0f2ae9f32781 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:37.714007 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c321e49-30a8-4e54-9513-fc1de968dbf6 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:37.714148 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f96eccd-edb1-4c9e-b183-0f2ae9f32781 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:37.714259 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c321e49-30a8-4e54-9513-fc1de968dbf6 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:37.714450 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c321e49-30a8-4e54-9513-fc1de968dbf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:37.740012 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-9f96eccd-edb1-4c9e-b183-0f2ae9f32781 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:25:37.740601 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-2c321e49-30a8-4e54-9513-fc1de968dbf6 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:25:37.741228 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-9f96eccd-edb1-4c9e-b183-0f2ae9f32781 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:25:37.741815 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-2c321e49-30a8-4e54-9513-fc1de968dbf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:25:37.750279 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 779/1548] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:25:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:25:37.750445 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 770/1549] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:25:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:25:37.754169 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4c1390-f34e-4d7a-a0f2-a5fd8bb0e114 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:37.754169 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4c1390-f34e-4d7a-a0f2-a5fd8bb0e114 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:37.754169 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4c1390-f34e-4d7a-a0f2-a5fd8bb0e114 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:37.754733 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c321e49-30a8-4e54-9513-fc1de968dbf6 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:25:37.765087 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c321e49-30a8-4e54-9513-fc1de968dbf6 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:25:37.770052 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee52f70-881d-424e-9359-f9fe0d0c2680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:25:37.770658 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee52f70-881d-424e-9359-f9fe0d0c2680 admin admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:37.770958 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee52f70-881d-424e-9359-f9fe0d0c2680 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:37.771234 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee52f70-881d-424e-9359-f9fe0d0c2680 admin admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:37.771871 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee52f70-881d-424e-9359-f9fe0d0c2680 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:25:37.773446 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee52f70-881d-424e-9359-f9fe0d0c2680 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:25:37.780168 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-1b4c1390-f34e-4d7a-a0f2-a5fd8bb0e114 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:25:37.781756 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-1b4c1390-f34e-4d7a-a0f2-a5fd8bb0e114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:25:37.784367 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 780/1550] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:25:37 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:25:37.788429 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283ae480-eedd-4a91-be30-5e64b95958ec None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:37.788719 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283ae480-eedd-4a91-be30-5e64b95958ec None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:37.788985 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283ae480-eedd-4a91-be30-5e64b95958ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:37.791430 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 771/1551] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:25:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:25:37.792933 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561281f9-5b65-4623-b611-195805dcb568 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:37.793218 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561281f9-5b65-4623-b611-195805dcb568 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:37.793544 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561281f9-5b65-4623-b611-195805dcb568 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:37.817899 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-283ae480-eedd-4a91-be30-5e64b95958ec None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:25:37.819123 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-283ae480-eedd-4a91-be30-5e64b95958ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:25:37.819551 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-561281f9-5b65-4623-b611-195805dcb568 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:25:37.820782 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-561281f9-5b65-4623-b611-195805dcb568 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:25:37.827516 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 781/1552] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:25:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:25:37.828765 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd61a469-0a7d-4cf2-9bd0-b58651073839 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:37.829063 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd61a469-0a7d-4cf2-9bd0-b58651073839 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:37.829330 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd61a469-0a7d-4cf2-9bd0-b58651073839 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:37.830229 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 772/1553] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:25:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:25:37.831095 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561281f9-5b65-4623-b611-195805dcb568 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:25:37.845121 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561281f9-5b65-4623-b611-195805dcb568 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:25:37.848973 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-cd61a469-0a7d-4cf2-9bd0-b58651073839 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:25:37.849710 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3307d28a-c227-4bd2-9a6f-607d1bed62d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:25:37.850161 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-cd61a469-0a7d-4cf2-9bd0-b58651073839 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:25:37.850315 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3307d28a-c227-4bd2-9a6f-607d1bed62d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:37.850595 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3307d28a-c227-4bd2-9a6f-607d1bed62d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:37.850863 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3307d28a-c227-4bd2-9a6f-607d1bed62d1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:37.851589 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3307d28a-c227-4bd2-9a6f-607d1bed62d1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:25:37.851889 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3307d28a-c227-4bd2-9a6f-607d1bed62d1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:25:37.853473 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3307d28a-c227-4bd2-9a6f-607d1bed62d1 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:25:37.857859 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 782/1554] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:25:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:25:37.859054 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e83105-3420-455f-862b-1ddb4a6c06e3 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:37.859335 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e83105-3420-455f-862b-1ddb4a6c06e3 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:37.859610 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e83105-3420-455f-862b-1ddb4a6c06e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:37.865768 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 773/1555] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 24 17:25:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:25:37.866570 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3307d28a-c227-4bd2-9a6f-607d1bed62d1 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:25:37.879137 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3307d28a-c227-4bd2-9a6f-607d1bed62d1 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:25:37.883772 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24f8b08-f0fd-453b-9d90-c32c8e0d0713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:25:37.883939 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-16e83105-3420-455f-862b-1ddb4a6c06e3 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:25:37.884362 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24f8b08-f0fd-453b-9d90-c32c8e0d0713 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:37.884640 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24f8b08-f0fd-453b-9d90-c32c8e0d0713 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:37.884912 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24f8b08-f0fd-453b-9d90-c32c8e0d0713 admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:37.885020 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-16e83105-3420-455f-862b-1ddb4a6c06e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:25:37.885543 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24f8b08-f0fd-453b-9d90-c32c8e0d0713 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:25:37.887188 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24f8b08-f0fd-453b-9d90-c32c8e0d0713 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:25:37.894131 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 783/1556] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:25:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:25:37.895367 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454fb541-5376-4d5c-b66b-347013abfa33 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:37.895666 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454fb541-5376-4d5c-b66b-347013abfa33 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:37.895907 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454fb541-5376-4d5c-b66b-347013abfa33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:37.898634 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 774/1557] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:25:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:25:37.899954 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcbe79f-ac26-4309-b54d-7e988feb94de None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:37.900242 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcbe79f-ac26-4309-b54d-7e988feb94de None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:37.900516 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcbe79f-ac26-4309-b54d-7e988feb94de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:37.916765 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-454fb541-5376-4d5c-b66b-347013abfa33 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:25:37.917897 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-454fb541-5376-4d5c-b66b-347013abfa33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:25:37.919602 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-9dcbe79f-ac26-4309-b54d-7e988feb94de None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:25:37.920762 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-9dcbe79f-ac26-4309-b54d-7e988feb94de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:25:37.925795 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 784/1558] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:25:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:25:37.926564 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454fb541-5376-4d5c-b66b-347013abfa33 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:25:37.928934 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 775/1559] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:25:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:25:37.932318 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d21a905-f43f-47af-a837-096de51f0452 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:37.932892 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d21a905-f43f-47af-a837-096de51f0452 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:37.933388 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d21a905-f43f-47af-a837-096de51f0452 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:37.935946 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454fb541-5376-4d5c-b66b-347013abfa33 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:25:37.940512 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602d77ae-9b7e-4c81-8e5b-c6afb040dae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:25:37.941105 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602d77ae-9b7e-4c81-8e5b-c6afb040dae1 admin admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:37.941379 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602d77ae-9b7e-4c81-8e5b-c6afb040dae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:37.941777 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602d77ae-9b7e-4c81-8e5b-c6afb040dae1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:37.942277 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602d77ae-9b7e-4c81-8e5b-c6afb040dae1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:25:37.943362 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602d77ae-9b7e-4c81-8e5b-c6afb040dae1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:25:37.943678 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: WARNING py.warnings [None req-602d77ae-9b7e-4c81-8e5b-c6afb040dae1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 17:25:37.943678 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: warnings.warn(msg) Aug 24 17:25:37.943678 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: Aug 24 17:25:37.944548 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602d77ae-9b7e-4c81-8e5b-c6afb040dae1 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:25:37.954103 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-8d21a905-f43f-47af-a837-096de51f0452 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:25:37.955367 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-8d21a905-f43f-47af-a837-096de51f0452 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:25:37.957090 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 785/1560] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 24 17:25:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:25:37.957940 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602d77ae-9b7e-4c81-8e5b-c6afb040dae1 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:25:37.963468 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 776/1561] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:25:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:25:37.964072 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d21a905-f43f-47af-a837-096de51f0452 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:25:37.971543 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602d77ae-9b7e-4c81-8e5b-c6afb040dae1 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:25:37.973957 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d21a905-f43f-47af-a837-096de51f0452 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:25:37.976366 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16e43cc-5e3c-40f4-8658-c62a80c61e65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:25:37.976966 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16e43cc-5e3c-40f4-8658-c62a80c61e65 admin admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:37.977249 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16e43cc-5e3c-40f4-8658-c62a80c61e65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:37.977521 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16e43cc-5e3c-40f4-8658-c62a80c61e65 admin admin] PATH_INFO: `/v3/domains` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:37.978188 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16e43cc-5e3c-40f4-8658-c62a80c61e65 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:25:37.978488 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16e43cc-5e3c-40f4-8658-c62a80c61e65 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:25:37.978731 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a10f207-3208-46b6-bf6a-0d88558eb431 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:25:37.979358 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a10f207-3208-46b6-bf6a-0d88558eb431 admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:37.979637 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a10f207-3208-46b6-bf6a-0d88558eb431 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:37.979907 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a10f207-3208-46b6-bf6a-0d88558eb431 admin admin] PATH_INFO: `/v3/projects` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:37.980100 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16e43cc-5e3c-40f4-8658-c62a80c61e65 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:25:37.980518 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a10f207-3208-46b6-bf6a-0d88558eb431 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:25:37.981550 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: WARNING py.warnings [None req-5a10f207-3208-46b6-bf6a-0d88558eb431 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 17:25:37.981550 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: warnings.warn(msg) Aug 24 17:25:37.981550 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: Aug 24 17:25:37.982713 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a10f207-3208-46b6-bf6a-0d88558eb431 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:25:37.992701 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 786/1562] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 24 17:25:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:25:37.993644 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16e43cc-5e3c-40f4-8658-c62a80c61e65 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:25:38.006852 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16e43cc-5e3c-40f4-8658-c62a80c61e65 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:25:38.015891 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd57bdb-829a-43e1-b807-5c7b5af15b6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:25:38.016772 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd57bdb-829a-43e1-b807-5c7b5af15b6b admin admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:38.016772 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd57bdb-829a-43e1-b807-5c7b5af15b6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:38.016772 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd57bdb-829a-43e1-b807-5c7b5af15b6b admin admin] PATH_INFO: `/v3/domains` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:38.017459 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd57bdb-829a-43e1-b807-5c7b5af15b6b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:25:38.017839 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd57bdb-829a-43e1-b807-5c7b5af15b6b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:25:38.019500 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd57bdb-829a-43e1-b807-5c7b5af15b6b admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:25:38.036065 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 787/1563] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 24 17:25:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:25:38.036894 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd57bdb-829a-43e1-b807-5c7b5af15b6b admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:25:38.048152 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd57bdb-829a-43e1-b807-5c7b5af15b6b admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:25:38.052845 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f504f84e-caaf-41d1-b8d4-17d7e5d0c909 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:25:38.053492 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f504f84e-caaf-41d1-b8d4-17d7e5d0c909 admin admin] REQUEST_METHOD: `PUT` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:38.053770 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f504f84e-caaf-41d1-b8d4-17d7e5d0c909 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:38.054041 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f504f84e-caaf-41d1-b8d4-17d7e5d0c909 admin admin] PATH_INFO: `/v3/projects/6c8612c530e1495f8d6f4927560a9012/users/ef9e96b709b345c08a4ea10f14211668/roles/60dc7e16aee746d5a3e8aa27d7d8525c` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:38.068272 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f504f84e-caaf-41d1-b8d4-17d7e5d0c909 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c8612c530e1495f8d6f4927560a9012, user_id=ef9e96b709b345c08a4ea10f14211668, role_id=60dc7e16aee746d5a3e8aa27d7d8525c)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:25:38.070896 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f504f84e-caaf-41d1-b8d4-17d7e5d0c909 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:25:38.105629 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 777/1564] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 24 17:25:37 2022] POST /identity/v3/projects => generated 393 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:25:38.106458 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a10f207-3208-46b6-bf6a-0d88558eb431 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:25:38.132519 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 788/1565] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 24 17:25:38 2022] PUT /identity/v3/projects/6c8612c530e1495f8d6f4927560a9012/users/ef9e96b709b345c08a4ea10f14211668/roles/60dc7e16aee746d5a3e8aa27d7d8525c => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:25:38.134205 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972cf602-5310-4fcf-8b3d-61e534bf6056 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:38.134476 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972cf602-5310-4fcf-8b3d-61e534bf6056 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:38.134750 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972cf602-5310-4fcf-8b3d-61e534bf6056 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:38.155207 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:25:38.155796 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-972cf602-5310-4fcf-8b3d-61e534bf6056 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:25:38.156616 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-972cf602-5310-4fcf-8b3d-61e534bf6056 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:25:38.161087 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:25:38.176405 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a10f207-3208-46b6-bf6a-0d88558eb431 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:25:38.214555 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 789/1566] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:25:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:25:38.215402 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972cf602-5310-4fcf-8b3d-61e534bf6056 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:25:38.225958 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972cf602-5310-4fcf-8b3d-61e534bf6056 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:25:38.230653 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b15fd1b-1cab-456a-ae95-0e09b10af7e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:25:38.237598 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17be69f8-b068-4b59-9178-592e77de4050 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:25:38.237751 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b15fd1b-1cab-456a-ae95-0e09b10af7e5 admin admin] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:38.237854 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17be69f8-b068-4b59-9178-592e77de4050 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:38.237958 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b15fd1b-1cab-456a-ae95-0e09b10af7e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:38.238057 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17be69f8-b068-4b59-9178-592e77de4050 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:38.238161 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b15fd1b-1cab-456a-ae95-0e09b10af7e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:38.238259 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17be69f8-b068-4b59-9178-592e77de4050 admin admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:38.238348 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b15fd1b-1cab-456a-ae95-0e09b10af7e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:25:38.238435 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17be69f8-b068-4b59-9178-592e77de4050 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:25:38.238435 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17be69f8-b068-4b59-9178-592e77de4050 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:25:38.238617 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b15fd1b-1cab-456a-ae95-0e09b10af7e5 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:25:38.238707 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17be69f8-b068-4b59-9178-592e77de4050 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:25:38.246188 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 778/1567] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 24 17:25:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:25:38.247588 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17be69f8-b068-4b59-9178-592e77de4050 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:25:38.263921 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17be69f8-b068-4b59-9178-592e77de4050 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:25:38.268884 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6912d22-a293-4ff1-9172-a2c266588e13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:25:38.269481 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6912d22-a293-4ff1-9172-a2c266588e13 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:38.269758 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6912d22-a293-4ff1-9172-a2c266588e13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:38.270029 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6912d22-a293-4ff1-9172-a2c266588e13 admin admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:38.270679 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6912d22-a293-4ff1-9172-a2c266588e13 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:25:38.271034 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6912d22-a293-4ff1-9172-a2c266588e13 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:25:38.272612 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6912d22-a293-4ff1-9172-a2c266588e13 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:25:38.285533 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 779/1568] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 24 17:25:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:25:38.289410 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6912d22-a293-4ff1-9172-a2c266588e13 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:25:38.295578 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6912d22-a293-4ff1-9172-a2c266588e13 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:25:38.300199 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd789088-5a0b-4ae7-8f96-e843d2ee83e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:25:38.300839 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd789088-5a0b-4ae7-8f96-e843d2ee83e7 admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:38.301156 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd789088-5a0b-4ae7-8f96-e843d2ee83e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:38.301486 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd789088-5a0b-4ae7-8f96-e843d2ee83e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:38.302405 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd789088-5a0b-4ae7-8f96-e843d2ee83e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:25:38.304355 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd789088-5a0b-4ae7-8f96-e843d2ee83e7 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:25:38.313095 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 790/1569] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 24 17:25:38 2022] POST /identity/v3/projects => generated 371 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:25:38.313963 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b15fd1b-1cab-456a-ae95-0e09b10af7e5 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:25:38.376503 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b15fd1b-1cab-456a-ae95-0e09b10af7e5 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:25:38.381914 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19632cd7-741d-41c4-ab8b-7ddfec256935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:25:38.382655 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19632cd7-741d-41c4-ab8b-7ddfec256935 admin admin] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:38.382973 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19632cd7-741d-41c4-ab8b-7ddfec256935 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:38.383257 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19632cd7-741d-41c4-ab8b-7ddfec256935 admin admin] PATH_INFO: `/v3/projects` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:38.383890 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19632cd7-741d-41c4-ab8b-7ddfec256935 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:25:38.385727 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 780/1570] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 24 17:25:38 2022] POST /identity/v3/projects => generated 387 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:25:38.385853 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19632cd7-741d-41c4-ab8b-7ddfec256935 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:25:38.386531 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd789088-5a0b-4ae7-8f96-e843d2ee83e7 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:25:38.449341 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd789088-5a0b-4ae7-8f96-e843d2ee83e7 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:25:38.454520 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04152ff1-5d18-4c58-ba69-c5a41cdfec2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:25:38.455159 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04152ff1-5d18-4c58-ba69-c5a41cdfec2c admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:38.455440 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04152ff1-5d18-4c58-ba69-c5a41cdfec2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:38.455710 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04152ff1-5d18-4c58-ba69-c5a41cdfec2c admin admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:38.456436 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04152ff1-5d18-4c58-ba69-c5a41cdfec2c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:25:38.462485 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04152ff1-5d18-4c58-ba69-c5a41cdfec2c admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:25:38.488214 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 791/1571] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 24 17:25:38 2022] POST /identity/v3/projects => generated 403 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:25:38.489463 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dd5821-4608-4d6c-82dd-574764e7c28f None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:38.490007 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dd5821-4608-4d6c-82dd-574764e7c28f None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:38.490294 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dd5821-4608-4d6c-82dd-574764e7c28f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:38.508804 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-a5dd5821-4608-4d6c-82dd-574764e7c28f None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:25:38.514152 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-a5dd5821-4608-4d6c-82dd-574764e7c28f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:25:38.569992 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 781/1572] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 24 17:25:38 2022] POST /identity/v3/users => generated 297 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:25:38.571077 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04152ff1-5d18-4c58-ba69-c5a41cdfec2c admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:25:38.573596 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 792/1573] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:25:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:25:38.574664 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dd5821-4608-4d6c-82dd-574764e7c28f None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:25:38.581639 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04152ff1-5d18-4c58-ba69-c5a41cdfec2c admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:25:38.584970 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dd5821-4608-4d6c-82dd-574764e7c28f None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:25:38.586702 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba6c18b-f0f8-4521-b9c8-89ab94ca494b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:25:38.587401 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba6c18b-f0f8-4521-b9c8-89ab94ca494b admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:38.587746 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba6c18b-f0f8-4521-b9c8-89ab94ca494b admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:38.588048 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba6c18b-f0f8-4521-b9c8-89ab94ca494b admin admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:38.588691 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba6c18b-f0f8-4521-b9c8-89ab94ca494b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:25:38.588986 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba6c18b-f0f8-4521-b9c8-89ab94ca494b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:25:38.589787 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faba6b53-13fe-4d74-b3cd-d0e662847b41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:25:38.590485 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faba6b53-13fe-4d74-b3cd-d0e662847b41 admin admin] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:38.590602 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba6c18b-f0f8-4521-b9c8-89ab94ca494b admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:25:38.590787 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faba6b53-13fe-4d74-b3cd-d0e662847b41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:38.591096 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faba6b53-13fe-4d74-b3cd-d0e662847b41 admin admin] PATH_INFO: `/v3/projects` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:38.591727 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faba6b53-13fe-4d74-b3cd-d0e662847b41 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:25:38.593762 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faba6b53-13fe-4d74-b3cd-d0e662847b41 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:25:38.603199 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 782/1574] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 24 17:25:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:25:38.604024 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba6c18b-f0f8-4521-b9c8-89ab94ca494b admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:25:38.611087 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:25:38.615594 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba6c18b-f0f8-4521-b9c8-89ab94ca494b admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:25:38.620610 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77aa556f-e11b-4d6d-ba82-80715b1f9e7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:25:38.621247 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aa556f-e11b-4d6d-ba82-80715b1f9e7c admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:38.621551 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aa556f-e11b-4d6d-ba82-80715b1f9e7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:38.622403 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aa556f-e11b-4d6d-ba82-80715b1f9e7c admin admin] PATH_INFO: `/v3/projects` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:38.623065 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77aa556f-e11b-4d6d-ba82-80715b1f9e7c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:25:38.631709 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77aa556f-e11b-4d6d-ba82-80715b1f9e7c admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:25:38.680068 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 793/1575] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 24 17:25:38 2022] POST /identity/v3/projects => generated 371 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:25:38.680875 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faba6b53-13fe-4d74-b3cd-d0e662847b41 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:25:38.730803 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 783/1576] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 24 17:25:38 2022] POST /identity/v3/projects => generated 383 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:25:38.732243 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77aa556f-e11b-4d6d-ba82-80715b1f9e7c admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:25:38.738005 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:25:38.744408 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faba6b53-13fe-4d74-b3cd-d0e662847b41 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:25:38.795713 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77aa556f-e11b-4d6d-ba82-80715b1f9e7c admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:25:38.800772 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4af07b0-ebea-4635-a322-911c143df671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:25:38.804119 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4af07b0-ebea-4635-a322-911c143df671 admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:38.804119 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4af07b0-ebea-4635-a322-911c143df671 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:38.804334 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44aa5d0e-f200-4c86-8df1-a03cd0c72546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:25:38.804459 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4af07b0-ebea-4635-a322-911c143df671 admin admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:38.804550 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44aa5d0e-f200-4c86-8df1-a03cd0c72546 admin admin] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:38.804638 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4af07b0-ebea-4635-a322-911c143df671 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:25:38.804724 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44aa5d0e-f200-4c86-8df1-a03cd0c72546 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:38.804724 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44aa5d0e-f200-4c86-8df1-a03cd0c72546 admin admin] PATH_INFO: `/v3/users` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:38.804724 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44aa5d0e-f200-4c86-8df1-a03cd0c72546 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:25:38.804983 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4af07b0-ebea-4635-a322-911c143df671 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:25:38.805070 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: WARNING py.warnings [None req-44aa5d0e-f200-4c86-8df1-a03cd0c72546 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 17:25:38.805070 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: warnings.warn(msg) Aug 24 17:25:38.805070 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: Aug 24 17:25:38.805721 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44aa5d0e-f200-4c86-8df1-a03cd0c72546 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:25:38.880858 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 784/1577] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 24 17:25:38 2022] POST /identity/v3/users => generated 294 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:25:38.882395 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4af07b0-ebea-4635-a322-911c143df671 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:25:38.893470 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4af07b0-ebea-4635-a322-911c143df671 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:25:38.893672 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 794/1578] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 24 17:25:38 2022] POST /identity/v3/users => generated 286 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:25:38.894345 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44aa5d0e-f200-4c86-8df1-a03cd0c72546 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:25:38.898488 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034c11c2-0da3-439e-b226-6a1ff8b1bd6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:25:38.899118 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034c11c2-0da3-439e-b226-6a1ff8b1bd6a admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:38.899399 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034c11c2-0da3-439e-b226-6a1ff8b1bd6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:38.899790 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034c11c2-0da3-439e-b226-6a1ff8b1bd6a admin admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:38.900413 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034c11c2-0da3-439e-b226-6a1ff8b1bd6a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:25:38.902330 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034c11c2-0da3-439e-b226-6a1ff8b1bd6a admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:25:38.904465 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44aa5d0e-f200-4c86-8df1-a03cd0c72546 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:25:38.909109 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90b9313-e300-497a-8083-1243562ac054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:25:38.913176 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90b9313-e300-497a-8083-1243562ac054 admin admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:38.913176 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90b9313-e300-497a-8083-1243562ac054 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:38.913176 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90b9313-e300-497a-8083-1243562ac054 admin admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:38.913176 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90b9313-e300-497a-8083-1243562ac054 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:25:38.913176 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90b9313-e300-497a-8083-1243562ac054 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:25:38.922721 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 795/1579] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:25:38 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:25:38.923732 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90b9313-e300-497a-8083-1243562ac054 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:25:38.978348 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90b9313-e300-497a-8083-1243562ac054 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:25:38.981981 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc6d756-412f-439e-86bf-64099316e4cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e9ed4d47aef4140bfb2d3bb798cc559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:25:38.982508 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc6d756-412f-439e-86bf-64099316e4cc service neutron] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:38.982782 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc6d756-412f-439e-86bf-64099316e4cc service neutron] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:38.983086 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc6d756-412f-439e-86bf-64099316e4cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:38.992047 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc6d756-412f-439e-86bf-64099316e4cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:25:38.993462 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc6d756-412f-439e-86bf-64099316e4cc service neutron] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:25:39.003618 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 796/1580] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:25:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:25:39.004317 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 785/1581] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 24 17:25:38 2022] POST /identity/v3/users => generated 302 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:25:39.004517 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc6d756-412f-439e-86bf-64099316e4cc service neutron] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:25:39.005428 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034c11c2-0da3-439e-b226-6a1ff8b1bd6a admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:25:39.015160 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc6d756-412f-439e-86bf-64099316e4cc service neutron] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:25:39.016251 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034c11c2-0da3-439e-b226-6a1ff8b1bd6a admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:25:39.020805 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c5729f-07a0-4233-8c1f-e5f8ef43ff63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:25:39.020975 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c5729f-07a0-4233-8c1f-e5f8ef43ff63 admin admin] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:39.021247 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c5729f-07a0-4233-8c1f-e5f8ef43ff63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:39.021521 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c5729f-07a0-4233-8c1f-e5f8ef43ff63 admin admin] PATH_INFO: `/v3/projects` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:39.021861 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7bab74-0071-4cf1-a370-5f3e0e8023ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:25:39.022137 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c5729f-07a0-4233-8c1f-e5f8ef43ff63 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:25:39.022800 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7bab74-0071-4cf1-a370-5f3e0e8023ea admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:39.023066 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7bab74-0071-4cf1-a370-5f3e0e8023ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:39.023443 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7bab74-0071-4cf1-a370-5f3e0e8023ea admin admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:39.024146 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7bab74-0071-4cf1-a370-5f3e0e8023ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:25:39.024351 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c5729f-07a0-4233-8c1f-e5f8ef43ff63 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:25:39.026267 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7bab74-0071-4cf1-a370-5f3e0e8023ea admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:25:39.136004 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:25:39.142182 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 786/1582] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 24 17:25:39 2022] POST /identity/v3/users => generated 286 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:25:39.146306 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7bab74-0071-4cf1-a370-5f3e0e8023ea admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:25:39.146442 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 797/1583] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 24 17:25:39 2022] POST /identity/v3/projects => generated 379 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:25:39.146528 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c5729f-07a0-4233-8c1f-e5f8ef43ff63 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:25:39.155205 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7bab74-0071-4cf1-a370-5f3e0e8023ea admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:25:39.214741 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c5729f-07a0-4233-8c1f-e5f8ef43ff63 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:25:39.216670 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67a9cdb-ad1c-4274-ba91-237bb030c161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:25:39.217316 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67a9cdb-ad1c-4274-ba91-237bb030c161 admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:39.217689 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67a9cdb-ad1c-4274-ba91-237bb030c161 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:39.217959 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67a9cdb-ad1c-4274-ba91-237bb030c161 admin admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:39.218625 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67a9cdb-ad1c-4274-ba91-237bb030c161 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:25:39.220044 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446f0dc6-e9da-49a2-b456-3be05d18513d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:25:39.220653 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446f0dc6-e9da-49a2-b456-3be05d18513d admin admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:39.220785 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67a9cdb-ad1c-4274-ba91-237bb030c161 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:25:39.220939 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446f0dc6-e9da-49a2-b456-3be05d18513d admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:39.221257 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446f0dc6-e9da-49a2-b456-3be05d18513d admin admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:39.221897 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446f0dc6-e9da-49a2-b456-3be05d18513d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:25:39.223515 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446f0dc6-e9da-49a2-b456-3be05d18513d admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:25:39.235280 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 798/1584] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:25:39 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:25:39.236044 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446f0dc6-e9da-49a2-b456-3be05d18513d admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:25:39.245366 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446f0dc6-e9da-49a2-b456-3be05d18513d admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:25:39.250016 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c71a952-9017-420b-b3dc-4b76a945b6a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:25:39.250588 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c71a952-9017-420b-b3dc-4b76a945b6a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:39.253515 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c71a952-9017-420b-b3dc-4b76a945b6a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:39.253515 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c71a952-9017-420b-b3dc-4b76a945b6a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:39.253515 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c71a952-9017-420b-b3dc-4b76a945b6a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:25:39.253515 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c71a952-9017-420b-b3dc-4b76a945b6a0 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:25:39.263480 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 799/1585] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:25:39 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:25:39.264400 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c71a952-9017-420b-b3dc-4b76a945b6a0 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:25:39.274130 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c71a952-9017-420b-b3dc-4b76a945b6a0 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:25:39.279149 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945ce2eb-cc53-43a7-be19-6c0275a46efd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:25:39.282233 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945ce2eb-cc53-43a7-be19-6c0275a46efd admin admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:39.282233 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945ce2eb-cc53-43a7-be19-6c0275a46efd admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:39.282233 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945ce2eb-cc53-43a7-be19-6c0275a46efd admin admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:39.282233 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945ce2eb-cc53-43a7-be19-6c0275a46efd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:25:39.282720 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945ce2eb-cc53-43a7-be19-6c0275a46efd admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:25:39.292197 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 800/1586] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:25:39 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:25:39.293121 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945ce2eb-cc53-43a7-be19-6c0275a46efd admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:25:39.303563 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945ce2eb-cc53-43a7-be19-6c0275a46efd admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:25:39.308517 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b5df3e-ebdb-4a8b-8d28-f4b55c35c979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:25:39.310735 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b5df3e-ebdb-4a8b-8d28-f4b55c35c979 admin admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:39.310735 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b5df3e-ebdb-4a8b-8d28-f4b55c35c979 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:39.310735 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b5df3e-ebdb-4a8b-8d28-f4b55c35c979 admin admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:39.310735 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b5df3e-ebdb-4a8b-8d28-f4b55c35c979 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:25:39.312064 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b5df3e-ebdb-4a8b-8d28-f4b55c35c979 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:25:39.322054 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 787/1587] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 24 17:25:39 2022] POST /identity/v3/users => generated 292 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:25:39.323706 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67a9cdb-ad1c-4274-ba91-237bb030c161 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:25:39.329896 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 801/1588] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:25:39 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:25:39.330819 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b5df3e-ebdb-4a8b-8d28-f4b55c35c979 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:25:39.339962 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67a9cdb-ad1c-4274-ba91-237bb030c161 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:25:39.343696 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b5df3e-ebdb-4a8b-8d28-f4b55c35c979 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:25:39.346666 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b862b7d3-1f94-449d-8d9b-f52e4787b08c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:25:39.347515 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b862b7d3-1f94-449d-8d9b-f52e4787b08c admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:39.347881 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b862b7d3-1f94-449d-8d9b-f52e4787b08c admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:39.348232 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b862b7d3-1f94-449d-8d9b-f52e4787b08c admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:39.348932 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b862b7d3-1f94-449d-8d9b-f52e4787b08c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:25:39.349258 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebed3c34-be6a-4c62-b239-2685f63b29ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:25:39.349998 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebed3c34-be6a-4c62-b239-2685f63b29ba admin admin] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:39.350363 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebed3c34-be6a-4c62-b239-2685f63b29ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:39.350692 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebed3c34-be6a-4c62-b239-2685f63b29ba admin admin] PATH_INFO: `/v3/users` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:39.351026 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b862b7d3-1f94-449d-8d9b-f52e4787b08c admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:25:39.351552 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebed3c34-be6a-4c62-b239-2685f63b29ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:25:39.354163 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebed3c34-be6a-4c62-b239-2685f63b29ba admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:25:39.365796 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 788/1589] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:25:39 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:25:39.365796 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b862b7d3-1f94-449d-8d9b-f52e4787b08c admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:25:39.375745 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b862b7d3-1f94-449d-8d9b-f52e4787b08c admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:25:39.382682 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb4a727-2e9e-4b82-9111-37f8ecd94b79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:25:39.385585 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb4a727-2e9e-4b82-9111-37f8ecd94b79 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:39.385585 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb4a727-2e9e-4b82-9111-37f8ecd94b79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:39.385869 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb4a727-2e9e-4b82-9111-37f8ecd94b79 admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:39.386589 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb4a727-2e9e-4b82-9111-37f8ecd94b79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:25:39.388852 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb4a727-2e9e-4b82-9111-37f8ecd94b79 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:25:39.402443 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 789/1590] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:25:39 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:25:39.403761 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb4a727-2e9e-4b82-9111-37f8ecd94b79 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:25:39.414827 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb4a727-2e9e-4b82-9111-37f8ecd94b79 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:25:39.420031 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67928481-de8e-480e-ad8d-2da496817d80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:25:39.420648 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67928481-de8e-480e-ad8d-2da496817d80 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:39.421068 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67928481-de8e-480e-ad8d-2da496817d80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:39.421344 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67928481-de8e-480e-ad8d-2da496817d80 admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:39.421984 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67928481-de8e-480e-ad8d-2da496817d80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:25:39.423620 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67928481-de8e-480e-ad8d-2da496817d80 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:25:39.433432 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 802/1591] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 24 17:25:39 2022] POST /identity/v3/users => generated 290 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:25:39.434563 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebed3c34-be6a-4c62-b239-2685f63b29ba admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:25:39.435141 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 790/1592] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:25:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:25:39.438855 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67928481-de8e-480e-ad8d-2da496817d80 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:25:39.447031 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebed3c34-be6a-4c62-b239-2685f63b29ba admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:25:39.448630 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67928481-de8e-480e-ad8d-2da496817d80 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:25:39.452561 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa74a95-5140-4d39-8fbd-a336f4faeb21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:25:39.460635 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa74a95-5140-4d39-8fbd-a336f4faeb21 admin admin] REQUEST_METHOD: `PUT` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:39.460635 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa74a95-5140-4d39-8fbd-a336f4faeb21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:39.460635 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa74a95-5140-4d39-8fbd-a336f4faeb21 admin admin] PATH_INFO: `/v3/projects/cf9b8b1e5c0f4021ae5f40c11c962222/users/0696e6f707274727b57a9a7493c47664/roles/60dc7e16aee746d5a3e8aa27d7d8525c` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:39.460926 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202a0bda-0ebb-4421-b9a5-45d5e69fbdd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:25:39.460926 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202a0bda-0ebb-4421-b9a5-45d5e69fbdd2 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:39.460926 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202a0bda-0ebb-4421-b9a5-45d5e69fbdd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:39.460926 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202a0bda-0ebb-4421-b9a5-45d5e69fbdd2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:39.461315 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202a0bda-0ebb-4421-b9a5-45d5e69fbdd2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:25:39.461315 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202a0bda-0ebb-4421-b9a5-45d5e69fbdd2 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:25:39.468416 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 791/1593] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:25:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:25:39.470009 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa74a95-5140-4d39-8fbd-a336f4faeb21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf9b8b1e5c0f4021ae5f40c11c962222, user_id=0696e6f707274727b57a9a7493c47664, role_id=60dc7e16aee746d5a3e8aa27d7d8525c)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:25:39.471900 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202a0bda-0ebb-4421-b9a5-45d5e69fbdd2 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:25:39.472895 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa74a95-5140-4d39-8fbd-a336f4faeb21 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:25:39.478129 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=88,keystone:INSERT=12 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:25:39.488008 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202a0bda-0ebb-4421-b9a5-45d5e69fbdd2 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:25:39.488008 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed27842-8643-4613-bc11-a191a1c804fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:25:39.488546 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed27842-8643-4613-bc11-a191a1c804fe admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:39.488821 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed27842-8643-4613-bc11-a191a1c804fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:39.489095 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed27842-8643-4613-bc11-a191a1c804fe admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:39.489729 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed27842-8643-4613-bc11-a191a1c804fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:25:39.491369 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed27842-8643-4613-bc11-a191a1c804fe admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:25:39.501725 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 792/1594] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:25:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:25:39.502663 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed27842-8643-4613-bc11-a191a1c804fe admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:25:39.512125 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed27842-8643-4613-bc11-a191a1c804fe admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:25:39.517744 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7c793e-f91d-4089-b712-57f2af204ccb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:25:39.518368 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7c793e-f91d-4089-b712-57f2af204ccb admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:39.518653 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7c793e-f91d-4089-b712-57f2af204ccb admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:39.518972 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7c793e-f91d-4089-b712-57f2af204ccb admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:39.519612 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7c793e-f91d-4089-b712-57f2af204ccb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:25:39.521199 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7c793e-f91d-4089-b712-57f2af204ccb admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:25:39.535318 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 793/1595] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:25:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:25:39.535318 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7c793e-f91d-4089-b712-57f2af204ccb admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:25:39.546993 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7c793e-f91d-4089-b712-57f2af204ccb admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:25:39.551451 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ca4c05-b98d-4e1a-9d54-a0965f6045c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:25:39.551911 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ca4c05-b98d-4e1a-9d54-a0965f6045c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:39.552210 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ca4c05-b98d-4e1a-9d54-a0965f6045c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:39.552485 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ca4c05-b98d-4e1a-9d54-a0965f6045c4 admin admin] PATH_INFO: `/v3/projects/15cff6e1349f4bb1a2905e6e5231fc2d/users/c379c845fccd43c08e8699976d506801/roles/60dc7e16aee746d5a3e8aa27d7d8525c` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:39.568125 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 803/1596] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 24 17:25:39 2022] PUT /identity/v3/projects/cf9b8b1e5c0f4021ae5f40c11c962222/users/0696e6f707274727b57a9a7493c47664/roles/60dc7e16aee746d5a3e8aa27d7d8525c => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:25:39.568125 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa74a95-5140-4d39-8fbd-a336f4faeb21 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:25:39.576731 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ca4c05-b98d-4e1a-9d54-a0965f6045c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=15cff6e1349f4bb1a2905e6e5231fc2d, user_id=c379c845fccd43c08e8699976d506801, role_id=60dc7e16aee746d5a3e8aa27d7d8525c)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:25:39.577924 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: WARNING py.warnings [None req-c5ca4c05-b98d-4e1a-9d54-a0965f6045c4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 17:25:39.577924 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: warnings.warn(msg) Aug 24 17:25:39.577924 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: Aug 24 17:25:39.580159 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ca4c05-b98d-4e1a-9d54-a0965f6045c4 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:25:39.643955 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa74a95-5140-4d39-8fbd-a336f4faeb21 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:25:39.651093 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6675ed6-89fc-464a-a50c-a367120e8e87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:25:39.653749 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6675ed6-89fc-464a-a50c-a367120e8e87 admin admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:39.653749 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6675ed6-89fc-464a-a50c-a367120e8e87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:39.653749 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6675ed6-89fc-464a-a50c-a367120e8e87 admin admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:39.660159 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6675ed6-89fc-464a-a50c-a367120e8e87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:25:39.660159 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6675ed6-89fc-464a-a50c-a367120e8e87 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:25:39.671060 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 804/1597] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:25:39 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:25:39.677642 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6675ed6-89fc-464a-a50c-a367120e8e87 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:25:39.684994 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 794/1598] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 24 17:25:39 2022] PUT /identity/v3/projects/15cff6e1349f4bb1a2905e6e5231fc2d/users/c379c845fccd43c08e8699976d506801/roles/60dc7e16aee746d5a3e8aa27d7d8525c => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:25:39.686353 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ca4c05-b98d-4e1a-9d54-a0965f6045c4 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:25:39.689694 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6675ed6-89fc-464a-a50c-a367120e8e87 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:25:39.707054 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:25:39.752088 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ca4c05-b98d-4e1a-9d54-a0965f6045c4 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:25:39.752534 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e62133e-e000-4a5f-8d9a-d1cdc44b24ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:25:39.753237 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e62133e-e000-4a5f-8d9a-d1cdc44b24ad admin admin] REQUEST_METHOD: `PUT` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:39.753518 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e62133e-e000-4a5f-8d9a-d1cdc44b24ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:39.753795 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e62133e-e000-4a5f-8d9a-d1cdc44b24ad admin admin] PATH_INFO: `/v3/projects/0ac70939c0ea4cf598ca556226001634/users/2996e61856294680b80dd1de0ed827fe/roles/60dc7e16aee746d5a3e8aa27d7d8525c` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:39.757152 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4d2cab-3706-4c15-a72d-958a2ac2dad1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:25:39.757763 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4d2cab-3706-4c15-a72d-958a2ac2dad1 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:39.758067 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4d2cab-3706-4c15-a72d-958a2ac2dad1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:39.758341 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4d2cab-3706-4c15-a72d-958a2ac2dad1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:39.759083 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4d2cab-3706-4c15-a72d-958a2ac2dad1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:25:39.760744 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4d2cab-3706-4c15-a72d-958a2ac2dad1 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:25:39.771280 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e62133e-e000-4a5f-8d9a-d1cdc44b24ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ac70939c0ea4cf598ca556226001634, user_id=2996e61856294680b80dd1de0ed827fe, role_id=60dc7e16aee746d5a3e8aa27d7d8525c)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:25:39.776299 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 795/1599] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:25:39 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:25:39.776299 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4d2cab-3706-4c15-a72d-958a2ac2dad1 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:25:39.776571 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e62133e-e000-4a5f-8d9a-d1cdc44b24ad admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:25:39.786656 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4d2cab-3706-4c15-a72d-958a2ac2dad1 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:25:39.791509 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2ccd76-09f1-479a-9a37-3be4390a2981 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:25:39.794311 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2ccd76-09f1-479a-9a37-3be4390a2981 admin admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:39.794311 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2ccd76-09f1-479a-9a37-3be4390a2981 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:39.794311 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2ccd76-09f1-479a-9a37-3be4390a2981 admin admin] PATH_INFO: `/v3/projects/d88bff95d6084fc3a2623ad6e4a58d68/users/0be478bd2b6046cd93c368f531be477c/roles/60dc7e16aee746d5a3e8aa27d7d8525c` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:39.810462 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2ccd76-09f1-479a-9a37-3be4390a2981 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d88bff95d6084fc3a2623ad6e4a58d68, user_id=0be478bd2b6046cd93c368f531be477c, role_id=60dc7e16aee746d5a3e8aa27d7d8525c)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:25:39.813161 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2ccd76-09f1-479a-9a37-3be4390a2981 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:25:39.905457 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 805/1600] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 24 17:25:39 2022] PUT /identity/v3/projects/0ac70939c0ea4cf598ca556226001634/users/2996e61856294680b80dd1de0ed827fe/roles/60dc7e16aee746d5a3e8aa27d7d8525c => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:25:39.906097 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 796/1601] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 24 17:25:39 2022] PUT /identity/v3/projects/d88bff95d6084fc3a2623ad6e4a58d68/users/0be478bd2b6046cd93c368f531be477c/roles/60dc7e16aee746d5a3e8aa27d7d8525c => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:25:39.906539 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e62133e-e000-4a5f-8d9a-d1cdc44b24ad admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:25:39.907904 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57a9605-e2f2-41d8-8a3f-346172da3e9b None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:39.908419 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57a9605-e2f2-41d8-8a3f-346172da3e9b None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:39.908869 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57a9605-e2f2-41d8-8a3f-346172da3e9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:39.936525 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-d57a9605-e2f2-41d8-8a3f-346172da3e9b None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:25:39.937755 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-d57a9605-e2f2-41d8-8a3f-346172da3e9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:25:39.992489 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e62133e-e000-4a5f-8d9a-d1cdc44b24ad admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:25:40.013040 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 797/1602] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:25:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:25:40.013040 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57a9605-e2f2-41d8-8a3f-346172da3e9b None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:25:40.019430 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57a9605-e2f2-41d8-8a3f-346172da3e9b None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:25:40.024444 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764428fb-2bf9-44de-be3b-e1af50349682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:25:40.025083 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764428fb-2bf9-44de-be3b-e1af50349682 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:40.025537 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764428fb-2bf9-44de-be3b-e1af50349682 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:40.025812 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764428fb-2bf9-44de-be3b-e1af50349682 admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:40.026464 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764428fb-2bf9-44de-be3b-e1af50349682 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:25:40.028170 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764428fb-2bf9-44de-be3b-e1af50349682 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:25:40.039316 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 798/1603] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:25:40 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:25:40.040725 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8978b3-4021-41df-8e0d-6f6d1ebdb4e0 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:40.041024 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8978b3-4021-41df-8e0d-6f6d1ebdb4e0 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:40.041328 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8978b3-4021-41df-8e0d-6f6d1ebdb4e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:40.067388 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-ed8978b3-4021-41df-8e0d-6f6d1ebdb4e0 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:25:40.069086 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-ed8978b3-4021-41df-8e0d-6f6d1ebdb4e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:25:40.074995 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb73f442-9de2-4b3f-965e-1b400c7e1201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:25:40.076135 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb73f442-9de2-4b3f-965e-1b400c7e1201 admin admin] REQUEST_METHOD: `PUT` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:40.076423 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb73f442-9de2-4b3f-965e-1b400c7e1201 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:40.076697 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb73f442-9de2-4b3f-965e-1b400c7e1201 admin admin] PATH_INFO: `/v3/projects/39e4a03cb9b04427b645ead2b53e1737/users/48a734287d7c46e984f750030bf52418/roles/60dc7e16aee746d5a3e8aa27d7d8525c` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:40.080444 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 799/1604] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:25:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:25:40.080444 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8978b3-4021-41df-8e0d-6f6d1ebdb4e0 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:25:40.089373 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8978b3-4021-41df-8e0d-6f6d1ebdb4e0 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:25:40.095593 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cee94d-bf5b-4db2-88af-cbcefa06e598 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:25:40.096275 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cee94d-bf5b-4db2-88af-cbcefa06e598 admin admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:40.096551 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cee94d-bf5b-4db2-88af-cbcefa06e598 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:40.096821 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cee94d-bf5b-4db2-88af-cbcefa06e598 admin admin] PATH_INFO: `/v3/projects/f6ebea743b3843f3b6274ddfc0b5b73a/users/160ce86c89ea449da346d98d20ce2a9e/roles/60dc7e16aee746d5a3e8aa27d7d8525c` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:40.100494 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb73f442-9de2-4b3f-965e-1b400c7e1201 admin admin] RBAC: Authorizing `identity:create_grant(project_id=39e4a03cb9b04427b645ead2b53e1737, user_id=48a734287d7c46e984f750030bf52418, role_id=60dc7e16aee746d5a3e8aa27d7d8525c)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:25:40.103513 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb73f442-9de2-4b3f-965e-1b400c7e1201 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:25:40.109226 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:25:40.113223 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cee94d-bf5b-4db2-88af-cbcefa06e598 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6ebea743b3843f3b6274ddfc0b5b73a, user_id=160ce86c89ea449da346d98d20ce2a9e, role_id=60dc7e16aee746d5a3e8aa27d7d8525c)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:25:40.115900 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cee94d-bf5b-4db2-88af-cbcefa06e598 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:25:40.207611 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 806/1605] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 24 17:25:39 2022] PUT /identity/v3/projects/39e4a03cb9b04427b645ead2b53e1737/users/48a734287d7c46e984f750030bf52418/roles/60dc7e16aee746d5a3e8aa27d7d8525c => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:25:40.208097 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 800/1606] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 24 17:25:40 2022] PUT /identity/v3/projects/f6ebea743b3843f3b6274ddfc0b5b73a/users/160ce86c89ea449da346d98d20ce2a9e/roles/60dc7e16aee746d5a3e8aa27d7d8525c => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:25:40.209027 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78bb808-bd91-4307-861a-f8e2ffa57040 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:40.209374 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78bb808-bd91-4307-861a-f8e2ffa57040 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:40.209479 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d27e5b-4386-4ef7-a484-f351d0ae5a57 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:40.209697 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d27e5b-4386-4ef7-a484-f351d0ae5a57 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:40.209942 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78bb808-bd91-4307-861a-f8e2ffa57040 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:40.210041 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d27e5b-4386-4ef7-a484-f351d0ae5a57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:40.236448 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-f78bb808-bd91-4307-861a-f8e2ffa57040 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:25:40.237683 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-f78bb808-bd91-4307-861a-f8e2ffa57040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:25:40.243338 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-67d27e5b-4386-4ef7-a484-f351d0ae5a57 None None] MFA Rules not processed for user `2996e61856294680b80dd1de0ed827fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:25:40.244524 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-67d27e5b-4386-4ef7-a484-f351d0ae5a57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:25:40.263356 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:25:40.302088 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 807/1607] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:25:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:25:40.303113 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78bb808-bd91-4307-861a-f8e2ffa57040 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:25:40.304386 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-f78bb808-bd91-4307-861a-f8e2ffa57040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:25:40.317166 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 801/1608] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:25:40 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:25:40.317166 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d27e5b-4386-4ef7-a484-f351d0ae5a57 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:25:40.339609 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d27e5b-4386-4ef7-a484-f351d0ae5a57 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:25:40.347989 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64a6c98-358d-4a9e-9378-0d15b3f60ccd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:25:40.348859 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64a6c98-358d-4a9e-9378-0d15b3f60ccd admin admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:40.349378 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64a6c98-358d-4a9e-9378-0d15b3f60ccd admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:40.349500 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64a6c98-358d-4a9e-9378-0d15b3f60ccd admin admin] PATH_INFO: `/v3/projects/879c5673d2cf49f5973d1faa5f774c43/users/bb2bc3ba87a549018d50fe3c9c6720d0/roles/60dc7e16aee746d5a3e8aa27d7d8525c` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:40.366863 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64a6c98-358d-4a9e-9378-0d15b3f60ccd admin admin] RBAC: Authorizing `identity:create_grant(project_id=879c5673d2cf49f5973d1faa5f774c43, user_id=bb2bc3ba87a549018d50fe3c9c6720d0, role_id=60dc7e16aee746d5a3e8aa27d7d8525c)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:25:40.370295 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64a6c98-358d-4a9e-9378-0d15b3f60ccd admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:25:40.380985 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78bb808-bd91-4307-861a-f8e2ffa57040 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:25:40.384651 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f8db48-7110-4bef-8e98-e10ae453945b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e9ed4d47aef4140bfb2d3bb798cc559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:25:40.385280 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f8db48-7110-4bef-8e98-e10ae453945b service neutron] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:40.385562 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f8db48-7110-4bef-8e98-e10ae453945b service neutron] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:40.385834 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f8db48-7110-4bef-8e98-e10ae453945b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:40.395962 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f8db48-7110-4bef-8e98-e10ae453945b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:25:40.397397 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f8db48-7110-4bef-8e98-e10ae453945b service neutron] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:25:40.407439 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 808/1609] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:25:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:25:40.408231 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f8db48-7110-4bef-8e98-e10ae453945b service neutron] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:25:40.416242 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f8db48-7110-4bef-8e98-e10ae453945b service neutron] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:25:40.420060 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb03631f-391d-4967-a85e-4cbcac0c15a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e9ed4d47aef4140bfb2d3bb798cc559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:25:40.420968 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb03631f-391d-4967-a85e-4cbcac0c15a3 service neutron] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:40.421106 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb03631f-391d-4967-a85e-4cbcac0c15a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:40.421307 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb03631f-391d-4967-a85e-4cbcac0c15a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:40.431921 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb03631f-391d-4967-a85e-4cbcac0c15a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:25:40.433448 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb03631f-391d-4967-a85e-4cbcac0c15a3 service neutron] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:25:40.444374 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 809/1610] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:25:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:25:40.446071 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80f9236-d0dd-4350-ac93-dcc2f5a73feb None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:40.446466 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80f9236-d0dd-4350-ac93-dcc2f5a73feb None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:40.446980 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80f9236-d0dd-4350-ac93-dcc2f5a73feb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:40.480544 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-e80f9236-d0dd-4350-ac93-dcc2f5a73feb None None] MFA Rules not processed for user `48a734287d7c46e984f750030bf52418`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:25:40.481940 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-e80f9236-d0dd-4350-ac93-dcc2f5a73feb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:25:40.502551 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 802/1611] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 24 17:25:40 2022] PUT /identity/v3/projects/879c5673d2cf49f5973d1faa5f774c43/users/bb2bc3ba87a549018d50fe3c9c6720d0/roles/60dc7e16aee746d5a3e8aa27d7d8525c => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:25:40.502551 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1855b670-a13c-45a7-a501-bbfc143b6df1 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:40.502551 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1855b670-a13c-45a7-a501-bbfc143b6df1 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:40.502551 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1855b670-a13c-45a7-a501-bbfc143b6df1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:40.524192 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-1855b670-a13c-45a7-a501-bbfc143b6df1 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:25:40.525373 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-1855b670-a13c-45a7-a501-bbfc143b6df1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:25:40.555168 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 810/1612] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:25:40 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:25:40.555168 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80f9236-d0dd-4350-ac93-dcc2f5a73feb None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:25:40.564048 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80f9236-d0dd-4350-ac93-dcc2f5a73feb None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:25:40.618141 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 803/1613] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:25:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:25:40.618359 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a82e0a-ea7b-43c6-9fde-f0e22ed46700 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e9ed4d47aef4140bfb2d3bb798cc559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:25:40.619065 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a82e0a-ea7b-43c6-9fde-f0e22ed46700 service neutron] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:40.619405 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a82e0a-ea7b-43c6-9fde-f0e22ed46700 service neutron] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:40.619947 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a82e0a-ea7b-43c6-9fde-f0e22ed46700 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:40.620496 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aba492-900d-442d-a5ab-318613bd0a8f None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:40.620955 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aba492-900d-442d-a5ab-318613bd0a8f None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:40.621306 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aba492-900d-442d-a5ab-318613bd0a8f None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:40.623624 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 804/1614] 10.222.0.18 () {54 vars in 962 bytes} [Wed Aug 24 17:25:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:25:40.625393 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31cd1dd-8045-46dd-8bf2-da2a7e474f26 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:40.625745 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31cd1dd-8045-46dd-8bf2-da2a7e474f26 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:40.626091 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31cd1dd-8045-46dd-8bf2-da2a7e474f26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:40.632912 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a82e0a-ea7b-43c6-9fde-f0e22ed46700 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:25:40.635303 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a82e0a-ea7b-43c6-9fde-f0e22ed46700 service neutron] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:25:40.648601 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 811/1615] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:25:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:25:40.649584 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a82e0a-ea7b-43c6-9fde-f0e22ed46700 service neutron] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:25:40.652602 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-f31cd1dd-8045-46dd-8bf2-da2a7e474f26 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:25:40.654255 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-f31cd1dd-8045-46dd-8bf2-da2a7e474f26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:25:40.660109 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a82e0a-ea7b-43c6-9fde-f0e22ed46700 service neutron] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:25:40.665322 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6d0615-741a-4288-999d-28c4298a3ff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:25:40.665961 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6d0615-741a-4288-999d-28c4298a3ff2 admin admin] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:40.666238 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6d0615-741a-4288-999d-28c4298a3ff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:40.666562 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6d0615-741a-4288-999d-28c4298a3ff2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:40.667282 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6d0615-741a-4288-999d-28c4298a3ff2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:25:40.669238 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 805/1616] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:25:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:25:40.669627 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6d0615-741a-4288-999d-28c4298a3ff2 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:25:40.671156 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7b9f91-97cd-4d7c-8f1e-5acd5a4c7b4f None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:40.671971 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7b9f91-97cd-4d7c-8f1e-5acd5a4c7b4f None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:40.671971 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7b9f91-97cd-4d7c-8f1e-5acd5a4c7b4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:40.719363 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-8f7b9f91-97cd-4d7c-8f1e-5acd5a4c7b4f None None] MFA Rules not processed for user `557aca9255cb4bbb8af52e796c5c5177`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:25:40.720506 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-8f7b9f91-97cd-4d7c-8f1e-5acd5a4c7b4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:25:40.759324 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:25:40.760209 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 812/1617] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 24 17:25:40 2022] POST /identity/v3/projects => generated 373 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:25:40.767233 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6d0615-741a-4288-999d-28c4298a3ff2 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:25:40.830098 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 806/1618] 10.222.0.18 () {58 vars in 1059 bytes} [Wed Aug 24 17:25:40 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:25:40.831181 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7b9f91-97cd-4d7c-8f1e-5acd5a4c7b4f None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:25:40.845236 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7b9f91-97cd-4d7c-8f1e-5acd5a4c7b4f None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:25:40.847498 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:25:40.852712 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcf7a9f-1ab3-4afe-8228-868158313a18 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e9ed4d47aef4140bfb2d3bb798cc559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:25:40.852712 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcf7a9f-1ab3-4afe-8228-868158313a18 service neutron] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:40.852712 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcf7a9f-1ab3-4afe-8228-868158313a18 service neutron] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:40.852712 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcf7a9f-1ab3-4afe-8228-868158313a18 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:40.854270 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6d0615-741a-4288-999d-28c4298a3ff2 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:25:40.859855 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb04c635-c880-4dde-8752-bb378926060d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e9ed4d47aef4140bfb2d3bb798cc559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:25:40.860622 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb04c635-c880-4dde-8752-bb378926060d service neutron] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:40.860867 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb04c635-c880-4dde-8752-bb378926060d service neutron] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:40.861196 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb04c635-c880-4dde-8752-bb378926060d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:40.954130 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb04c635-c880-4dde-8752-bb378926060d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:25:40.956594 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb04c635-c880-4dde-8752-bb378926060d service neutron] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:25:40.963941 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcf7a9f-1ab3-4afe-8228-868158313a18 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:25:40.966659 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcf7a9f-1ab3-4afe-8228-868158313a18 service neutron] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:25:40.971662 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 813/1619] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:25:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:25:40.984575 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 807/1620] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:25:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:25:40.987216 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb04c635-c880-4dde-8752-bb378926060d service neutron] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:25:40.987567 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcf7a9f-1ab3-4afe-8228-868158313a18 service neutron] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:25:41.017539 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb04c635-c880-4dde-8752-bb378926060d service neutron] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:25:41.032270 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4e6477-7801-46be-8189-337d37470770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:25:41.032991 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4e6477-7801-46be-8189-337d37470770 admin admin] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:41.033430 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4e6477-7801-46be-8189-337d37470770 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:41.033689 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4e6477-7801-46be-8189-337d37470770 admin admin] PATH_INFO: `/v3/users` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:41.034466 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4e6477-7801-46be-8189-337d37470770 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:25:41.036489 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4e6477-7801-46be-8189-337d37470770 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:25:41.109524 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcf7a9f-1ab3-4afe-8228-868158313a18 service neutron] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:25:41.116574 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b37860-4cc5-4692-8628-9b96577b4ab0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557aca9255cb4bbb8af52e796c5c5177', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:25:41.117348 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b37860-4cc5-4692-8628-9b96577b4ab0 service nova] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:41.117582 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b37860-4cc5-4692-8628-9b96577b4ab0 service nova] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:41.117914 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b37860-4cc5-4692-8628-9b96577b4ab0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:41.141169 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 814/1621] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 24 17:25:40 2022] POST /identity/v3/users => generated 293 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:25:41.142780 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4e6477-7801-46be-8189-337d37470770 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:25:41.168624 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4e6477-7801-46be-8189-337d37470770 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:25:41.183642 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c96546-7d87-4f33-ae60-17178bc0813e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:25:41.185337 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c96546-7d87-4f33-ae60-17178bc0813e admin admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:41.185756 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c96546-7d87-4f33-ae60-17178bc0813e admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:41.186415 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c96546-7d87-4f33-ae60-17178bc0813e admin admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:41.187255 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c96546-7d87-4f33-ae60-17178bc0813e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:25:41.189370 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c96546-7d87-4f33-ae60-17178bc0813e admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:25:41.196269 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b37860-4cc5-4692-8628-9b96577b4ab0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:25:41.198954 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b37860-4cc5-4692-8628-9b96577b4ab0 service nova] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:25:41.206322 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 815/1622] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:25:41 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:25:41.207743 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c96546-7d87-4f33-ae60-17178bc0813e admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:25:41.220041 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 808/1623] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:25:40 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:25:41.224242 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c96546-7d87-4f33-ae60-17178bc0813e admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:25:41.227664 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdd8fdf-0329-4068-88f1-33691b66b934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:25:41.228604 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdd8fdf-0329-4068-88f1-33691b66b934 admin admin] REQUEST_METHOD: `PUT` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:41.228951 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdd8fdf-0329-4068-88f1-33691b66b934 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:41.229899 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdd8fdf-0329-4068-88f1-33691b66b934 admin admin] PATH_INFO: `/v3/projects/ffe6cef3506e425a954e1ef513fc57a3/users/22db250e5f7e4f2495818ed3494149ea/roles/c5346bc54abf445f9900edf94264c850` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:41.252440 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdd8fdf-0329-4068-88f1-33691b66b934 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ffe6cef3506e425a954e1ef513fc57a3, user_id=22db250e5f7e4f2495818ed3494149ea, role_id=c5346bc54abf445f9900edf94264c850)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:25:41.256146 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdd8fdf-0329-4068-88f1-33691b66b934 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:25:41.301309 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:25:41.363996 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 816/1624] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 24 17:25:41 2022] PUT /identity/v3/projects/ffe6cef3506e425a954e1ef513fc57a3/users/22db250e5f7e4f2495818ed3494149ea/roles/c5346bc54abf445f9900edf94264c850 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:25:41.368720 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b37860-4cc5-4692-8628-9b96577b4ab0 service nova] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:25:41.385618 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b37860-4cc5-4692-8628-9b96577b4ab0 service nova] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:25:41.487505 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487dfcbc-63b9-4aff-a58e-c3ab7e5ea144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:25:41.499531 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487dfcbc-63b9-4aff-a58e-c3ab7e5ea144 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:41.500112 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487dfcbc-63b9-4aff-a58e-c3ab7e5ea144 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:41.500607 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487dfcbc-63b9-4aff-a58e-c3ab7e5ea144 admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:41.501541 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487dfcbc-63b9-4aff-a58e-c3ab7e5ea144 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:25:41.503772 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487dfcbc-63b9-4aff-a58e-c3ab7e5ea144 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:25:41.529634 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 809/1625] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:25:41 2022] GET /identity/v3/roles => generated 1420 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:25:41.533911 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdd8fdf-0329-4068-88f1-33691b66b934 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:25:41.546797 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdd8fdf-0329-4068-88f1-33691b66b934 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:25:41.558266 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e5b016-4849-41e1-9f74-587883d62a6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:25:41.562748 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e5b016-4849-41e1-9f74-587883d62a6a admin admin] REQUEST_METHOD: `PUT` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:41.562748 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e5b016-4849-41e1-9f74-587883d62a6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:41.563702 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e5b016-4849-41e1-9f74-587883d62a6a admin admin] PATH_INFO: `/v3/projects/ffe6cef3506e425a954e1ef513fc57a3/users/22db250e5f7e4f2495818ed3494149ea/roles/c5346bc54abf445f9900edf94264c850` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:41.568797 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e5b016-4849-41e1-9f74-587883d62a6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ffe6cef3506e425a954e1ef513fc57a3, user_id=22db250e5f7e4f2495818ed3494149ea, role_id=c5346bc54abf445f9900edf94264c850)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:25:41.576130 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e5b016-4849-41e1-9f74-587883d62a6a admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:25:41.691602 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27761636-d317-4e1c-9860-42708f36c1d6 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:41.691602 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27761636-d317-4e1c-9860-42708f36c1d6 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:41.691602 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27761636-d317-4e1c-9860-42708f36c1d6 None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:41.691602 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 810/1627] 10.222.0.18 () {54 vars in 912 bytes} [Wed Aug 24 17:25:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:25:41.691602 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd3db14-b845-4c1e-9997-ad63d7823340 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:41.691602 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd3db14-b845-4c1e-9997-ad63d7823340 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:41.693484 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd3db14-b845-4c1e-9997-ad63d7823340 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:41.693670 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 817/1627] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 24 17:25:41 2022] PUT /identity/v3/projects/ffe6cef3506e425a954e1ef513fc57a3/users/22db250e5f7e4f2495818ed3494149ea/roles/c5346bc54abf445f9900edf94264c850 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:25:41.693670 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8388bb-bd5d-4341-90c4-f2a898e6450c None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:41.693670 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8388bb-bd5d-4341-90c4-f2a898e6450c None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:41.693670 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8388bb-bd5d-4341-90c4-f2a898e6450c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:41.693670 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-7b8388bb-bd5d-4341-90c4-f2a898e6450c None None] MFA Rules not processed for user `557aca9255cb4bbb8af52e796c5c5177`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:25:41.695371 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-7b8388bb-bd5d-4341-90c4-f2a898e6450c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:25:41.700168 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-bbd3db14-b845-4c1e-9997-ad63d7823340 None None] MFA Rules not processed for user `22db250e5f7e4f2495818ed3494149ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:25:41.701779 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-bbd3db14-b845-4c1e-9997-ad63d7823340 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:25:41.719178 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:25:41.825264 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 811/1628] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:25:41 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:25:41.844755 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd3db14-b845-4c1e-9997-ad63d7823340 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:25:41.865439 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 818/1629] 10.222.0.18 () {58 vars in 1009 bytes} [Wed Aug 24 17:25:41 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:25:41.874371 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8388bb-bd5d-4341-90c4-f2a898e6450c None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:25:41.946559 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd3db14-b845-4c1e-9997-ad63d7823340 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:25:41.953863 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fef32e2-19bc-4f47-bd37-e2b0d20d16d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be06246d47584b529150ceb98a5cfd95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:25:41.954525 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fef32e2-19bc-4f47-bd37-e2b0d20d16d7 service cinder] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:41.955022 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fef32e2-19bc-4f47-bd37-e2b0d20d16d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:41.955311 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fef32e2-19bc-4f47-bd37-e2b0d20d16d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:41.964655 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8388bb-bd5d-4341-90c4-f2a898e6450c None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:25:41.967963 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fef32e2-19bc-4f47-bd37-e2b0d20d16d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:25:41.970414 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49ec744-68bb-4289-9f0e-cb011eae5bf4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be06246d47584b529150ceb98a5cfd95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:25:41.970575 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fef32e2-19bc-4f47-bd37-e2b0d20d16d7 service cinder] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:25:41.971642 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49ec744-68bb-4289-9f0e-cb011eae5bf4 service cinder] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:41.972184 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49ec744-68bb-4289-9f0e-cb011eae5bf4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:41.972671 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49ec744-68bb-4289-9f0e-cb011eae5bf4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:42.001863 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49ec744-68bb-4289-9f0e-cb011eae5bf4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:25:42.001989 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 812/1630] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:25:41 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:25:42.004110 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49ec744-68bb-4289-9f0e-cb011eae5bf4 service cinder] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:25:42.052658 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 819/1631] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:25:41 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:25:43.028615 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce2317b-e267-4fac-a29e-0dcc238bd596 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:43.029129 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce2317b-e267-4fac-a29e-0dcc238bd596 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:43.029129 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce2317b-e267-4fac-a29e-0dcc238bd596 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:43.073350 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-5ce2317b-e267-4fac-a29e-0dcc238bd596 None None] MFA Rules not processed for user `0be478bd2b6046cd93c368f531be477c`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:25:43.074725 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-5ce2317b-e267-4fac-a29e-0dcc238bd596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:25:43.085713 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:25:43.216578 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 813/1632] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:25:43 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:25:43.220664 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49ec744-68bb-4289-9f0e-cb011eae5bf4 service cinder] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:25:43.384079 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49ec744-68bb-4289-9f0e-cb011eae5bf4 service cinder] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:25:43.397096 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce2317b-e267-4fac-a29e-0dcc238bd596 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:25:43.398047 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7e7306-7953-4e21-b13c-9376cb778f54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:25:43.398894 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7e7306-7953-4e21-b13c-9376cb778f54 admin admin] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:43.399759 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7e7306-7953-4e21-b13c-9376cb778f54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:43.400329 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7e7306-7953-4e21-b13c-9376cb778f54 admin admin] PATH_INFO: `/v3/projects` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:43.401112 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7e7306-7953-4e21-b13c-9376cb778f54 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:25:43.406851 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7e7306-7953-4e21-b13c-9376cb778f54 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:25:43.420390 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce2317b-e267-4fac-a29e-0dcc238bd596 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:25:43.436826 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43172002-dc59-41ea-ac96-f445638cc187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:25:43.438857 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43172002-dc59-41ea-ac96-f445638cc187 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:43.440188 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43172002-dc59-41ea-ac96-f445638cc187 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:43.441825 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43172002-dc59-41ea-ac96-f445638cc187 admin admin] PATH_INFO: `/v3/users/48a734287d7c46e984f750030bf52418` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:43.751665 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43172002-dc59-41ea-ac96-f445638cc187 admin admin] RBAC: Authorizing `identity:delete_user(user_id=48a734287d7c46e984f750030bf52418)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:25:43.751665 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: WARNING py.warnings [None req-43172002-dc59-41ea-ac96-f445638cc187 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 17:25:43.751665 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: warnings.warn(msg) Aug 24 17:25:43.751665 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: Aug 24 17:25:43.751665 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43172002-dc59-41ea-ac96-f445638cc187 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:25:43.754856 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 820/1633] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 24 17:25:43 2022] POST /identity/v3/projects => generated 403 bytes in 350 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:25:43.754856 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7e7306-7953-4e21-b13c-9376cb778f54 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:25:43.754856 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:25:43.754856 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7e7306-7953-4e21-b13c-9376cb778f54 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:25:43.757349 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d4b9f8-24bd-4ccd-9736-9f988f913b5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:25:43.757349 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d4b9f8-24bd-4ccd-9736-9f988f913b5b admin admin] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:43.757349 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d4b9f8-24bd-4ccd-9736-9f988f913b5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:43.757349 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d4b9f8-24bd-4ccd-9736-9f988f913b5b admin admin] PATH_INFO: `/v3/users` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:43.759259 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d4b9f8-24bd-4ccd-9736-9f988f913b5b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:25:43.759259 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d4b9f8-24bd-4ccd-9736-9f988f913b5b admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:25:43.973127 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 821/1634] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 24 17:25:43 2022] POST /identity/v3/users => generated 308 bytes in 399 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:25:43.974032 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb84ebf-301a-4c90-9b50-773de173d3a8 None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:43.974321 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb84ebf-301a-4c90-9b50-773de173d3a8 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:43.974603 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb84ebf-301a-4c90-9b50-773de173d3a8 None None] PATH_INFO: `/` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:43.980659 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 822/1635] 10.222.0.18 () {54 vars in 962 bytes} [Wed Aug 24 17:25:43 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:25:43.982110 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb84ebf-301a-4c90-9b50-773de173d3a8 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:25:43.998696 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb84ebf-301a-4c90-9b50-773de173d3a8 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:25:44.003911 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31cbd94-82ce-4d0d-bb47-73723dc667ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:25:44.004544 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31cbd94-82ce-4d0d-bb47-73723dc667ae admin admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:44.004828 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31cbd94-82ce-4d0d-bb47-73723dc667ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:44.005108 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31cbd94-82ce-4d0d-bb47-73723dc667ae admin admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:44.005765 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31cbd94-82ce-4d0d-bb47-73723dc667ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:25:44.007416 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31cbd94-82ce-4d0d-bb47-73723dc667ae admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:25:44.012674 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-43172002-dc59-41ea-ac96-f445638cc187 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '48a734287d7c46e984f750030bf52418'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:25:44.026631 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 823/1636] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:25:43 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:25:44.028127 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8eb6ac-f721-49ef-876c-13e6621a1e17 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:44.028417 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8eb6ac-f721-49ef-876c-13e6621a1e17 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:44.028697 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8eb6ac-f721-49ef-876c-13e6621a1e17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:44.071908 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-be8eb6ac-f721-49ef-876c-13e6621a1e17 None None] MFA Rules not processed for user `557aca9255cb4bbb8af52e796c5c5177`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:25:44.073209 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-be8eb6ac-f721-49ef-876c-13e6621a1e17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:25:44.129854 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-43172002-dc59-41ea-ac96-f445638cc187 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '48a734287d7c46e984f750030bf52418'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:25:44.131313 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-43172002-dc59-41ea-ac96-f445638cc187 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '48a734287d7c46e984f750030bf52418'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:25:44.151832 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-43172002-dc59-41ea-ac96-f445638cc187 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '48a734287d7c46e984f750030bf52418'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:25:44.152643 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-43172002-dc59-41ea-ac96-f445638cc187 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '48a734287d7c46e984f750030bf52418'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:25:44.211290 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 814/1637] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 24 17:25:43 2022] DELETE /identity/v3/users/48a734287d7c46e984f750030bf52418 => generated 0 bytes in 813 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:25:44.212725 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43172002-dc59-41ea-ac96-f445638cc187 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:25:44.220835 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-43172002-dc59-41ea-ac96-f445638cc187 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:25:44.230291 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 824/1638] 10.222.0.18 () {58 vars in 1059 bytes} [Wed Aug 24 17:25:44 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:25:44.230291 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33659326-5b4d-4aa3-984e-cb05a7a33ddc None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:44.230291 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33659326-5b4d-4aa3-984e-cb05a7a33ddc None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:44.230291 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33659326-5b4d-4aa3-984e-cb05a7a33ddc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:44.282930 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-33659326-5b4d-4aa3-984e-cb05a7a33ddc None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:25:44.288583 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-33659326-5b4d-4aa3-984e-cb05a7a33ddc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:25:44.372835 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43172002-dc59-41ea-ac96-f445638cc187 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:25:44.384474 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b85083e-3364-4470-94ae-c061e46f9f22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:25:44.386067 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b85083e-3364-4470-94ae-c061e46f9f22 admin admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:44.386722 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b85083e-3364-4470-94ae-c061e46f9f22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:44.388513 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b85083e-3364-4470-94ae-c061e46f9f22 admin admin] PATH_INFO: `/v3/projects/b9fd422cf1b6454cb92194f346a07830/users/389fce298f2c42cabe4ceca307b38151/roles/c5346bc54abf445f9900edf94264c850` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:44.410396 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 825/1639] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:25:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:25:44.412253 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7e6e66-f9c0-4320-85aa-b8e751d4398d None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:44.413133 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7e6e66-f9c0-4320-85aa-b8e751d4398d None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:44.413596 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7e6e66-f9c0-4320-85aa-b8e751d4398d None None] PATH_INFO: `/` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:44.415741 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 826/1640] 10.222.0.18 () {54 vars in 912 bytes} [Wed Aug 24 17:25:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:25:44.423122 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681e0c7c-d44a-4a99-9f5d-faeaa52d73fa None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:44.423693 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681e0c7c-d44a-4a99-9f5d-faeaa52d73fa None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:44.425010 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681e0c7c-d44a-4a99-9f5d-faeaa52d73fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:44.439816 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b85083e-3364-4470-94ae-c061e46f9f22 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9fd422cf1b6454cb92194f346a07830, user_id=389fce298f2c42cabe4ceca307b38151, role_id=c5346bc54abf445f9900edf94264c850)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:25:44.445088 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b85083e-3364-4470-94ae-c061e46f9f22 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:25:44.471426 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-681e0c7c-d44a-4a99-9f5d-faeaa52d73fa None None] MFA Rules not processed for user `557aca9255cb4bbb8af52e796c5c5177`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:25:44.473048 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-681e0c7c-d44a-4a99-9f5d-faeaa52d73fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:25:44.563061 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 815/1641] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 24 17:25:44 2022] PUT /identity/v3/projects/b9fd422cf1b6454cb92194f346a07830/users/389fce298f2c42cabe4ceca307b38151/roles/c5346bc54abf445f9900edf94264c850 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:25:44.563959 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b85083e-3364-4470-94ae-c061e46f9f22 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:25:44.567462 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-9b85083e-3364-4470-94ae-c061e46f9f22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:25:44.588473 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:25:44.633078 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 827/1642] 10.222.0.18 () {58 vars in 1009 bytes} [Wed Aug 24 17:25:44 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:25:44.638004 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681e0c7c-d44a-4a99-9f5d-faeaa52d73fa None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:25:44.681233 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681e0c7c-d44a-4a99-9f5d-faeaa52d73fa None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:25:44.744574 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b85083e-3364-4470-94ae-c061e46f9f22 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:25:44.750023 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f66af9-f123-4310-828a-36beb2fe73b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e9ed4d47aef4140bfb2d3bb798cc559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:25:44.754103 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f66af9-f123-4310-828a-36beb2fe73b2 service neutron] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:44.754103 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f66af9-f123-4310-828a-36beb2fe73b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:44.754103 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f66af9-f123-4310-828a-36beb2fe73b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:44.852011 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bcb633-3bb9-4254-b545-b9e4d1bca57c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:25:44.855610 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bcb633-3bb9-4254-b545-b9e4d1bca57c admin admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:44.857099 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bcb633-3bb9-4254-b545-b9e4d1bca57c admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:44.857478 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bcb633-3bb9-4254-b545-b9e4d1bca57c admin admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:44.858472 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bcb633-3bb9-4254-b545-b9e4d1bca57c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:25:44.863611 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bcb633-3bb9-4254-b545-b9e4d1bca57c admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:25:44.880321 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f66af9-f123-4310-828a-36beb2fe73b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:25:44.883389 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 828/1643] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:25:44 2022] GET /identity/v3/roles => generated 1420 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:25:44.884359 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bcb633-3bb9-4254-b545-b9e4d1bca57c admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:25:44.885346 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f66af9-f123-4310-828a-36beb2fe73b2 service neutron] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:25:44.886368 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-23bcb633-3bb9-4254-b545-b9e4d1bca57c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:25:44.908353 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 816/1644] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:25:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:25:44.910280 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f66af9-f123-4310-828a-36beb2fe73b2 service neutron] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:25:44.932124 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f66af9-f123-4310-828a-36beb2fe73b2 service neutron] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:25:44.945457 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bbf413-03d6-4e90-956e-5d6d2439e721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:25:44.946151 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bbf413-03d6-4e90-956e-5d6d2439e721 admin admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:44.946438 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bbf413-03d6-4e90-956e-5d6d2439e721 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:44.946722 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bbf413-03d6-4e90-956e-5d6d2439e721 admin admin] PATH_INFO: `/v3/projects/b9fd422cf1b6454cb92194f346a07830/users/389fce298f2c42cabe4ceca307b38151/roles/c5346bc54abf445f9900edf94264c850` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:44.950861 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bbf413-03d6-4e90-956e-5d6d2439e721 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9fd422cf1b6454cb92194f346a07830, user_id=389fce298f2c42cabe4ceca307b38151, role_id=c5346bc54abf445f9900edf94264c850)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:25:44.954197 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bbf413-03d6-4e90-956e-5d6d2439e721 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:25:44.980561 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 817/1645] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 24 17:25:44 2022] PUT /identity/v3/projects/b9fd422cf1b6454cb92194f346a07830/users/389fce298f2c42cabe4ceca307b38151/roles/c5346bc54abf445f9900edf94264c850 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:25:45.000363 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bcb633-3bb9-4254-b545-b9e4d1bca57c admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:25:45.110319 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44699000-1b35-4ee3-9f69-75ca7d577b4e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be06246d47584b529150ceb98a5cfd95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:25:45.111446 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44699000-1b35-4ee3-9f69-75ca7d577b4e service cinder] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:45.111788 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44699000-1b35-4ee3-9f69-75ca7d577b4e service cinder] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:45.112115 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44699000-1b35-4ee3-9f69-75ca7d577b4e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:45.136211 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:25:45.221519 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44699000-1b35-4ee3-9f69-75ca7d577b4e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:25:45.223154 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44699000-1b35-4ee3-9f69-75ca7d577b4e service cinder] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:25:45.235616 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 829/1646] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:25:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:25:46.197519 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bbf413-03d6-4e90-956e-5d6d2439e721 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:25:46.199626 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-12bbf413-03d6-4e90-956e-5d6d2439e721 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:25:46.217517 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:25:46.326367 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bbf413-03d6-4e90-956e-5d6d2439e721 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:25:46.333245 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df29461-418d-49fd-8dd9-b3e6847538b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:25:46.333995 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df29461-418d-49fd-8dd9-b3e6847538b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:46.337419 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df29461-418d-49fd-8dd9-b3e6847538b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:46.337870 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df29461-418d-49fd-8dd9-b3e6847538b1 admin admin] PATH_INFO: `/v3/projects/39e4a03cb9b04427b645ead2b53e1737` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:46.339497 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df29461-418d-49fd-8dd9-b3e6847538b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=39e4a03cb9b04427b645ead2b53e1737)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:25:46.343045 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df29461-418d-49fd-8dd9-b3e6847538b1 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:25:46.592618 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:25:46.601114 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-1df29461-418d-49fd-8dd9-b3e6847538b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39e4a03cb9b04427b645ead2b53e1737', 'request_id': 'req-1df29461-418d-49fd-8dd9-b3e6847538b1'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:25:46.734984 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-1df29461-418d-49fd-8dd9-b3e6847538b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39e4a03cb9b04427b645ead2b53e1737', 'request_id': 'req-1df29461-418d-49fd-8dd9-b3e6847538b1'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:25:46.747367 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-1df29461-418d-49fd-8dd9-b3e6847538b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39e4a03cb9b04427b645ead2b53e1737', 'request_id': 'req-1df29461-418d-49fd-8dd9-b3e6847538b1'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:25:46.777878 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-1df29461-418d-49fd-8dd9-b3e6847538b1 admin admin] The token cache is being invalidate because project 39e4a03cb9b04427b645ead2b53e1737 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81732) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:25:46.778879 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-1df29461-418d-49fd-8dd9-b3e6847538b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:25:46.780396 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-1df29461-418d-49fd-8dd9-b3e6847538b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:25:46.783264 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 818/1647] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 24 17:25:46 2022] DELETE /identity/v3/projects/39e4a03cb9b04427b645ead2b53e1737 => generated 0 bytes in 587 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:25:46.787063 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44699000-1b35-4ee3-9f69-75ca7d577b4e service cinder] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:25:46.801648 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44699000-1b35-4ee3-9f69-75ca7d577b4e service cinder] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:25:46.961752 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a024a47b-87c1-4cb5-9535-84c3b2cafbb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:25:46.962730 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a024a47b-87c1-4cb5-9535-84c3b2cafbb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:46.962834 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a024a47b-87c1-4cb5-9535-84c3b2cafbb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:46.964915 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a024a47b-87c1-4cb5-9535-84c3b2cafbb2 admin admin] PATH_INFO: `/v3/users/22db250e5f7e4f2495818ed3494149ea` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:46.964915 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a024a47b-87c1-4cb5-9535-84c3b2cafbb2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=22db250e5f7e4f2495818ed3494149ea)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:25:46.967572 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a024a47b-87c1-4cb5-9535-84c3b2cafbb2 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:25:47.252962 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-a024a47b-87c1-4cb5-9535-84c3b2cafbb2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '22db250e5f7e4f2495818ed3494149ea'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:25:47.325221 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-a024a47b-87c1-4cb5-9535-84c3b2cafbb2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '22db250e5f7e4f2495818ed3494149ea'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:25:47.403607 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-a024a47b-87c1-4cb5-9535-84c3b2cafbb2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '22db250e5f7e4f2495818ed3494149ea'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:25:47.404464 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-a024a47b-87c1-4cb5-9535-84c3b2cafbb2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '22db250e5f7e4f2495818ed3494149ea'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:25:47.420041 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-a024a47b-87c1-4cb5-9535-84c3b2cafbb2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '22db250e5f7e4f2495818ed3494149ea'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:25:47.441622 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 830/1648] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 24 17:25:46 2022] DELETE /identity/v3/users/22db250e5f7e4f2495818ed3494149ea => generated 0 bytes in 655 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:25:48.270627 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df29461-418d-49fd-8dd9-b3e6847538b1 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:25:48.272379 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-1df29461-418d-49fd-8dd9-b3e6847538b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:25:48.314102 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df29461-418d-49fd-8dd9-b3e6847538b1 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:25:48.477445 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70eec4f2-5a95-431d-8c84-b7301efa3d1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:25:48.479000 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eec4f2-5a95-431d-8c84-b7301efa3d1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:48.479380 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eec4f2-5a95-431d-8c84-b7301efa3d1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:48.479726 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eec4f2-5a95-431d-8c84-b7301efa3d1f admin admin] PATH_INFO: `/v3/users/2996e61856294680b80dd1de0ed827fe` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:48.481563 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70eec4f2-5a95-431d-8c84-b7301efa3d1f admin admin] RBAC: Authorizing `identity:delete_user(user_id=2996e61856294680b80dd1de0ed827fe)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:25:48.485651 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70eec4f2-5a95-431d-8c84-b7301efa3d1f admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:25:48.772366 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-70eec4f2-5a95-431d-8c84-b7301efa3d1f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2996e61856294680b80dd1de0ed827fe'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:25:48.870422 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-70eec4f2-5a95-431d-8c84-b7301efa3d1f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2996e61856294680b80dd1de0ed827fe'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:25:48.871926 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-70eec4f2-5a95-431d-8c84-b7301efa3d1f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2996e61856294680b80dd1de0ed827fe'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:25:48.900093 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-70eec4f2-5a95-431d-8c84-b7301efa3d1f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2996e61856294680b80dd1de0ed827fe'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:25:48.900897 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-70eec4f2-5a95-431d-8c84-b7301efa3d1f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2996e61856294680b80dd1de0ed827fe'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:25:48.952247 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 819/1649] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 24 17:25:48 2022] DELETE /identity/v3/users/2996e61856294680b80dd1de0ed827fe => generated 0 bytes in 682 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:25:48.958040 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aad36d9-8526-4821-a751-3a0b89fbd56f None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:48.958613 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aad36d9-8526-4821-a751-3a0b89fbd56f None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:48.958613 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aad36d9-8526-4821-a751-3a0b89fbd56f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:49.016146 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-9aad36d9-8526-4821-a751-3a0b89fbd56f None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:25:49.017460 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-9aad36d9-8526-4821-a751-3a0b89fbd56f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:25:49.142131 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:25:49.197937 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 831/1650] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:25:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:25:49.243830 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70eec4f2-5a95-431d-8c84-b7301efa3d1f admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:25:49.259730 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-70eec4f2-5a95-431d-8c84-b7301efa3d1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:25:49.489218 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aad36d9-8526-4821-a751-3a0b89fbd56f None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:25:49.490781 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-9aad36d9-8526-4821-a751-3a0b89fbd56f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:25:49.516256 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aad36d9-8526-4821-a751-3a0b89fbd56f None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:25:49.526488 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70eec4f2-5a95-431d-8c84-b7301efa3d1f admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:25:49.532223 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47f64c5-4382-47de-ba84-899b0b08be7f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e9ed4d47aef4140bfb2d3bb798cc559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:25:49.533349 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47f64c5-4382-47de-ba84-899b0b08be7f service neutron] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:49.533885 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47f64c5-4382-47de-ba84-899b0b08be7f service neutron] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:49.534259 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47f64c5-4382-47de-ba84-899b0b08be7f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:49.535238 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bac3f13-0831-4042-8f79-eceaae39ef37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:25:49.537388 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bac3f13-0831-4042-8f79-eceaae39ef37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:49.537388 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bac3f13-0831-4042-8f79-eceaae39ef37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:49.540183 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bac3f13-0831-4042-8f79-eceaae39ef37 admin admin] PATH_INFO: `/v3/projects/ffe6cef3506e425a954e1ef513fc57a3` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:49.549232 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:25:49.553630 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bac3f13-0831-4042-8f79-eceaae39ef37 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ffe6cef3506e425a954e1ef513fc57a3)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:25:49.557014 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47f64c5-4382-47de-ba84-899b0b08be7f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:25:49.559675 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47f64c5-4382-47de-ba84-899b0b08be7f service neutron] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:25:49.562948 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: WARNING py.warnings [None req-9bac3f13-0831-4042-8f79-eceaae39ef37 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 17:25:49.562948 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: warnings.warn(msg) Aug 24 17:25:49.562948 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: Aug 24 17:25:49.572179 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bac3f13-0831-4042-8f79-eceaae39ef37 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:25:49.582242 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 820/1651] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:25:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:25:49.805584 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a719797-afe2-4b40-bd28-dc2a85b8d022 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:49.806073 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a719797-afe2-4b40-bd28-dc2a85b8d022 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:49.807147 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a719797-afe2-4b40-bd28-dc2a85b8d022 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:49.854110 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-9bac3f13-0831-4042-8f79-eceaae39ef37 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ffe6cef3506e425a954e1ef513fc57a3', 'request_id': 'req-9bac3f13-0831-4042-8f79-eceaae39ef37'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:25:49.936227 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-9bac3f13-0831-4042-8f79-eceaae39ef37 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ffe6cef3506e425a954e1ef513fc57a3', 'request_id': 'req-9bac3f13-0831-4042-8f79-eceaae39ef37'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:25:49.962407 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-9bac3f13-0831-4042-8f79-eceaae39ef37 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ffe6cef3506e425a954e1ef513fc57a3', 'request_id': 'req-9bac3f13-0831-4042-8f79-eceaae39ef37'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:25:49.976232 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-8a719797-afe2-4b40-bd28-dc2a85b8d022 None None] MFA Rules not processed for user `389fce298f2c42cabe4ceca307b38151`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:25:49.985394 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-8a719797-afe2-4b40-bd28-dc2a85b8d022 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:25:49.998315 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-9bac3f13-0831-4042-8f79-eceaae39ef37 admin admin] The token cache is being invalidate because project ffe6cef3506e425a954e1ef513fc57a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81731) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:25:49.999984 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-9bac3f13-0831-4042-8f79-eceaae39ef37 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:25:50.002261 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-9bac3f13-0831-4042-8f79-eceaae39ef37 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:25:50.006514 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 832/1652] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 24 17:25:49 2022] DELETE /identity/v3/projects/ffe6cef3506e425a954e1ef513fc57a3 => generated 0 bytes in 518 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:25:50.012882 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7ffc53-33e8-46c2-8f07-87a6f742a0d4 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:50.013713 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7ffc53-33e8-46c2-8f07-87a6f742a0d4 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:50.014614 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7ffc53-33e8-46c2-8f07-87a6f742a0d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:50.070203 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-8c7ffc53-33e8-46c2-8f07-87a6f742a0d4 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:25:50.076060 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-8c7ffc53-33e8-46c2-8f07-87a6f742a0d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:25:50.204745 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 821/1653] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:25:49 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 402 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:25:50.258970 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 833/1654] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:25:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:25:50.259657 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46c1963-cab2-4ccb-aba1-e68f63e352d8 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:50.260184 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46c1963-cab2-4ccb-aba1-e68f63e352d8 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:50.260547 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46c1963-cab2-4ccb-aba1-e68f63e352d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:50.292895 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-d46c1963-cab2-4ccb-aba1-e68f63e352d8 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:25:50.295907 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-d46c1963-cab2-4ccb-aba1-e68f63e352d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:25:50.312592 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 822/1655] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:25:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:25:50.318888 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7ffc53-33e8-46c2-8f07-87a6f742a0d4 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:25:50.338381 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7ffc53-33e8-46c2-8f07-87a6f742a0d4 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:25:50.349692 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676bdd48-b064-43cf-b0d4-aef04f0ca844 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:25:50.351619 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676bdd48-b064-43cf-b0d4-aef04f0ca844 admin admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:50.352494 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676bdd48-b064-43cf-b0d4-aef04f0ca844 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:50.352789 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676bdd48-b064-43cf-b0d4-aef04f0ca844 admin admin] PATH_INFO: `/v3/domains` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:50.355434 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676bdd48-b064-43cf-b0d4-aef04f0ca844 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:25:50.359858 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676bdd48-b064-43cf-b0d4-aef04f0ca844 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:25:50.359858 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676bdd48-b064-43cf-b0d4-aef04f0ca844 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:25:50.382244 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46c1963-cab2-4ccb-aba1-e68f63e352d8 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:25:50.387087 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 834/1656] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 24 17:25:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:25:50.395293 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46c1963-cab2-4ccb-aba1-e68f63e352d8 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:25:50.420526 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02333c1-b875-4cb3-ac12-c8582dd27edc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:25:50.421680 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02333c1-b875-4cb3-ac12-c8582dd27edc admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:50.422179 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02333c1-b875-4cb3-ac12-c8582dd27edc admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:50.422648 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02333c1-b875-4cb3-ac12-c8582dd27edc admin admin] PATH_INFO: `/v3/projects` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:50.423538 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02333c1-b875-4cb3-ac12-c8582dd27edc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:25:50.425729 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02333c1-b875-4cb3-ac12-c8582dd27edc admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:25:50.566865 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 823/1657] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 24 17:25:50 2022] POST /identity/v3/projects => generated 363 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:25:50.568582 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676bdd48-b064-43cf-b0d4-aef04f0ca844 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:25:50.593446 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676bdd48-b064-43cf-b0d4-aef04f0ca844 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:25:50.788756 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5669b98-af88-4239-a0a7-6061e7ee7f45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:25:50.789684 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5669b98-af88-4239-a0a7-6061e7ee7f45 admin admin] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:50.790233 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5669b98-af88-4239-a0a7-6061e7ee7f45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:50.790580 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5669b98-af88-4239-a0a7-6061e7ee7f45 admin admin] PATH_INFO: `/v3/users` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:50.791387 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5669b98-af88-4239-a0a7-6061e7ee7f45 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:25:50.801292 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5669b98-af88-4239-a0a7-6061e7ee7f45 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:25:50.832561 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:25:50.929809 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 835/1658] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 24 17:25:50 2022] POST /identity/v3/users => generated 282 bytes in 359 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:25:50.931127 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02333c1-b875-4cb3-ac12-c8582dd27edc admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:25:50.950930 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02333c1-b875-4cb3-ac12-c8582dd27edc admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:25:50.963108 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bc65c0-af28-44fd-85fa-0da3497f670f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:25:50.963988 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bc65c0-af28-44fd-85fa-0da3497f670f admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:50.965230 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bc65c0-af28-44fd-85fa-0da3497f670f admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:50.966543 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bc65c0-af28-44fd-85fa-0da3497f670f admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:50.971208 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bc65c0-af28-44fd-85fa-0da3497f670f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:25:50.979366 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bc65c0-af28-44fd-85fa-0da3497f670f admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:25:51.026605 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 824/1659] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:25:50 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:25:51.026940 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5669b98-af88-4239-a0a7-6061e7ee7f45 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:25:51.044839 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5669b98-af88-4239-a0a7-6061e7ee7f45 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:25:51.055626 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af08f0ee-c713-4cf4-90c6-c53b1d9d10be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:25:51.055626 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af08f0ee-c713-4cf4-90c6-c53b1d9d10be admin admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:51.055626 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af08f0ee-c713-4cf4-90c6-c53b1d9d10be admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:51.055626 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af08f0ee-c713-4cf4-90c6-c53b1d9d10be admin admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:51.055916 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af08f0ee-c713-4cf4-90c6-c53b1d9d10be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:25:51.059969 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af08f0ee-c713-4cf4-90c6-c53b1d9d10be admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:25:51.087561 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 836/1660] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:25:51 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:25:51.091924 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bc65c0-af28-44fd-85fa-0da3497f670f admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:25:51.102294 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bc65c0-af28-44fd-85fa-0da3497f670f admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:25:51.119649 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0f2d0e-87f6-473b-9760-27da343bcbb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:25:51.120399 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0f2d0e-87f6-473b-9760-27da343bcbb8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:51.120681 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0f2d0e-87f6-473b-9760-27da343bcbb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:51.120957 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0f2d0e-87f6-473b-9760-27da343bcbb8 admin admin] PATH_INFO: `/v3/projects/5239fb586bd043c394710fa651720dbc/users/fda5136183484d6496da30da3d55fe4d/roles/60dc7e16aee746d5a3e8aa27d7d8525c` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:51.150573 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0f2d0e-87f6-473b-9760-27da343bcbb8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5239fb586bd043c394710fa651720dbc, user_id=fda5136183484d6496da30da3d55fe4d, role_id=60dc7e16aee746d5a3e8aa27d7d8525c)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:25:51.154369 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0f2d0e-87f6-473b-9760-27da343bcbb8 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:25:51.265892 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 825/1661] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 24 17:25:51 2022] PUT /identity/v3/projects/5239fb586bd043c394710fa651720dbc/users/fda5136183484d6496da30da3d55fe4d/roles/60dc7e16aee746d5a3e8aa27d7d8525c => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:25:51.276030 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b862e8-72f0-4622-b6da-7709226e29de None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:51.278937 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b862e8-72f0-4622-b6da-7709226e29de None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:51.278937 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b862e8-72f0-4622-b6da-7709226e29de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:51.349058 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-47b862e8-72f0-4622-b6da-7709226e29de None None] MFA Rules not processed for user `fda5136183484d6496da30da3d55fe4d`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:25:51.350537 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-47b862e8-72f0-4622-b6da-7709226e29de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:25:51.518566 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 837/1662] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:25:51 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:25:51.522838 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0f2d0e-87f6-473b-9760-27da343bcbb8 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:25:51.669523 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0f2d0e-87f6-473b-9760-27da343bcbb8 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:25:51.681962 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1356a0a6-0837-4e5d-808c-aed02e676e3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:25:51.682624 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1356a0a6-0837-4e5d-808c-aed02e676e3e admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:51.682943 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1356a0a6-0837-4e5d-808c-aed02e676e3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:51.683205 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1356a0a6-0837-4e5d-808c-aed02e676e3e admin admin] PATH_INFO: `/v3/projects` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:51.684397 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1356a0a6-0837-4e5d-808c-aed02e676e3e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:25:51.694825 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1356a0a6-0837-4e5d-808c-aed02e676e3e admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:25:51.813517 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 826/1663] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 24 17:25:51 2022] POST /identity/v3/projects => generated 365 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:25:51.819343 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b862e8-72f0-4622-b6da-7709226e29de None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:25:51.836708 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b862e8-72f0-4622-b6da-7709226e29de None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:25:51.970978 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bfc973-186d-45f4-a14d-175a3ca603e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:25:51.971448 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bfc973-186d-45f4-a14d-175a3ca603e6 admin admin] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:51.971808 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bfc973-186d-45f4-a14d-175a3ca603e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:51.972398 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bfc973-186d-45f4-a14d-175a3ca603e6 admin admin] PATH_INFO: `/v3/users` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:51.973184 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bfc973-186d-45f4-a14d-175a3ca603e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:25:51.977663 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bfc973-186d-45f4-a14d-175a3ca603e6 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:25:52.087576 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1356a0a6-0837-4e5d-808c-aed02e676e3e admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:25:52.089441 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-1356a0a6-0837-4e5d-808c-aed02e676e3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:25:52.097483 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:25:52.114147 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 838/1664] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 24 17:25:51 2022] POST /identity/v3/users => generated 289 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:25:52.114662 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1356a0a6-0837-4e5d-808c-aed02e676e3e admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:25:52.115945 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bfc973-186d-45f4-a14d-175a3ca603e6 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:25:52.129781 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621ffa91-973c-4436-b7be-77e148907797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:25:52.130171 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621ffa91-973c-4436-b7be-77e148907797 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:52.130460 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621ffa91-973c-4436-b7be-77e148907797 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:52.131405 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621ffa91-973c-4436-b7be-77e148907797 admin admin] PATH_INFO: `/v3/projects/0ac70939c0ea4cf598ca556226001634` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:52.133876 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621ffa91-973c-4436-b7be-77e148907797 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ac70939c0ea4cf598ca556226001634)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:25:52.135476 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bfc973-186d-45f4-a14d-175a3ca603e6 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:25:52.137253 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621ffa91-973c-4436-b7be-77e148907797 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:25:52.147685 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ce5e8f-2649-42a4-8066-ed6aac1fafa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:25:52.148317 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ce5e8f-2649-42a4-8066-ed6aac1fafa1 admin admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:52.148617 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ce5e8f-2649-42a4-8066-ed6aac1fafa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:52.148869 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ce5e8f-2649-42a4-8066-ed6aac1fafa1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:52.149535 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ce5e8f-2649-42a4-8066-ed6aac1fafa1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:25:52.151661 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ce5e8f-2649-42a4-8066-ed6aac1fafa1 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:25:52.182747 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 839/1665] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:25:52 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:25:52.187242 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ce5e8f-2649-42a4-8066-ed6aac1fafa1 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:25:52.199641 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ce5e8f-2649-42a4-8066-ed6aac1fafa1 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:25:52.215674 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b266ce37-073a-4089-a9ae-48344ab74404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:25:52.216352 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b266ce37-073a-4089-a9ae-48344ab74404 admin admin] REQUEST_METHOD: `PUT` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:52.216628 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b266ce37-073a-4089-a9ae-48344ab74404 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:52.216902 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b266ce37-073a-4089-a9ae-48344ab74404 admin admin] PATH_INFO: `/v3/projects/09d618690e4b4a12b0f0f06ea369f96c/users/fffd224b28f9487d8652d4183e4a2d5d/roles/c5346bc54abf445f9900edf94264c850` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:52.250560 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b266ce37-073a-4089-a9ae-48344ab74404 admin admin] RBAC: Authorizing `identity:create_grant(project_id=09d618690e4b4a12b0f0f06ea369f96c, user_id=fffd224b28f9487d8652d4183e4a2d5d, role_id=c5346bc54abf445f9900edf94264c850)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:25:52.255261 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b266ce37-073a-4089-a9ae-48344ab74404 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:25:52.368718 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 840/1666] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 24 17:25:52 2022] PUT /identity/v3/projects/09d618690e4b4a12b0f0f06ea369f96c/users/fffd224b28f9487d8652d4183e4a2d5d/roles/c5346bc54abf445f9900edf94264c850 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:25:52.373353 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b266ce37-073a-4089-a9ae-48344ab74404 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:25:52.396840 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-621ffa91-973c-4436-b7be-77e148907797 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ac70939c0ea4cf598ca556226001634', 'request_id': 'req-621ffa91-973c-4436-b7be-77e148907797'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:25:52.428033 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:INSERT=7,keystone:SELECT=93 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:25:52.486948 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-621ffa91-973c-4436-b7be-77e148907797 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ac70939c0ea4cf598ca556226001634', 'request_id': 'req-621ffa91-973c-4436-b7be-77e148907797'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:25:52.496048 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b266ce37-073a-4089-a9ae-48344ab74404 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:25:52.505128 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-621ffa91-973c-4436-b7be-77e148907797 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ac70939c0ea4cf598ca556226001634', 'request_id': 'req-621ffa91-973c-4436-b7be-77e148907797'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:25:52.550195 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-621ffa91-973c-4436-b7be-77e148907797 admin admin] The token cache is being invalidate because project 0ac70939c0ea4cf598ca556226001634 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81732) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:25:52.550710 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-621ffa91-973c-4436-b7be-77e148907797 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:25:52.552240 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-621ffa91-973c-4436-b7be-77e148907797 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:25:52.559355 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 827/1667] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 24 17:25:52 2022] DELETE /identity/v3/projects/0ac70939c0ea4cf598ca556226001634 => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:25:52.564586 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee48c3d-3a21-4d50-80da-6ae4eb9e888d None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:52.569834 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee48c3d-3a21-4d50-80da-6ae4eb9e888d None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:52.570941 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee48c3d-3a21-4d50-80da-6ae4eb9e888d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:52.631505 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-cee48c3d-3a21-4d50-80da-6ae4eb9e888d None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:25:52.639113 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-cee48c3d-3a21-4d50-80da-6ae4eb9e888d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:25:52.656309 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7355e0d2-df84-41ed-96b9-f0536d8757fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:25:52.658168 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7355e0d2-df84-41ed-96b9-f0536d8757fe admin admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:52.660316 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7355e0d2-df84-41ed-96b9-f0536d8757fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:52.660447 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7355e0d2-df84-41ed-96b9-f0536d8757fe admin admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:52.662316 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7355e0d2-df84-41ed-96b9-f0536d8757fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:25:52.666330 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7355e0d2-df84-41ed-96b9-f0536d8757fe admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:25:52.682700 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 828/1668] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:25:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:25:52.685726 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56859a6-2eed-4411-a5d3-e4e03d072143 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:52.686424 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56859a6-2eed-4411-a5d3-e4e03d072143 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:52.691333 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56859a6-2eed-4411-a5d3-e4e03d072143 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:52.704642 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 841/1669] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:25:52 2022] GET /identity/v3/roles => generated 1420 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:25:52.707480 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7355e0d2-df84-41ed-96b9-f0536d8757fe admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:25:52.710322 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-7355e0d2-df84-41ed-96b9-f0536d8757fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:25:52.732562 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-c56859a6-2eed-4411-a5d3-e4e03d072143 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:25:52.734090 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-c56859a6-2eed-4411-a5d3-e4e03d072143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:25:52.781053 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7355e0d2-df84-41ed-96b9-f0536d8757fe admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:25:52.791300 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 829/1670] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:25:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:25:52.792863 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f1fffb-f50b-4877-8969-5008e027fec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:25:52.793017 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56859a6-2eed-4411-a5d3-e4e03d072143 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:25:52.794540 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f1fffb-f50b-4877-8969-5008e027fec8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:52.795029 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f1fffb-f50b-4877-8969-5008e027fec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:52.795447 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f1fffb-f50b-4877-8969-5008e027fec8 admin admin] PATH_INFO: `/v3/projects/09d618690e4b4a12b0f0f06ea369f96c/users/fffd224b28f9487d8652d4183e4a2d5d/roles/c5346bc54abf445f9900edf94264c850` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:52.798694 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f1fffb-f50b-4877-8969-5008e027fec8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=09d618690e4b4a12b0f0f06ea369f96c, user_id=fffd224b28f9487d8652d4183e4a2d5d, role_id=c5346bc54abf445f9900edf94264c850)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:25:52.802638 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f1fffb-f50b-4877-8969-5008e027fec8 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:25:52.809610 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56859a6-2eed-4411-a5d3-e4e03d072143 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:25:52.832487 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 842/1671] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 24 17:25:52 2022] PUT /identity/v3/projects/09d618690e4b4a12b0f0f06ea369f96c/users/fffd224b28f9487d8652d4183e4a2d5d/roles/c5346bc54abf445f9900edf94264c850 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:25:52.834285 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba9d273-1e89-4a29-9fee-957be8e7c032 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:52.834688 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba9d273-1e89-4a29-9fee-957be8e7c032 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:52.835233 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba9d273-1e89-4a29-9fee-957be8e7c032 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:52.836544 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d95279-b4f4-413c-a315-100aea9f686b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:25:52.837179 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d95279-b4f4-413c-a315-100aea9f686b admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:52.837458 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d95279-b4f4-413c-a315-100aea9f686b admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:52.837727 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d95279-b4f4-413c-a315-100aea9f686b admin admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:52.838383 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d95279-b4f4-413c-a315-100aea9f686b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:25:52.839014 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d95279-b4f4-413c-a315-100aea9f686b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:25:52.840397 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d95279-b4f4-413c-a315-100aea9f686b admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:25:52.882206 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 830/1672] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 24 17:25:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:25:52.883414 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d95279-b4f4-413c-a315-100aea9f686b admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:25:52.895916 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d95279-b4f4-413c-a315-100aea9f686b admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:25:52.907558 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-fba9d273-1e89-4a29-9fee-957be8e7c032 None None] MFA Rules not processed for user `fffd224b28f9487d8652d4183e4a2d5d`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:25:52.908946 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-fba9d273-1e89-4a29-9fee-957be8e7c032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:25:53.046090 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af396d0-6c71-45e5-b1a9-83611324b087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:25:53.047330 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af396d0-6c71-45e5-b1a9-83611324b087 admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:53.048154 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af396d0-6c71-45e5-b1a9-83611324b087 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:53.049312 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af396d0-6c71-45e5-b1a9-83611324b087 admin admin] PATH_INFO: `/v3/projects` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:53.051581 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af396d0-6c71-45e5-b1a9-83611324b087 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:25:53.055940 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af396d0-6c71-45e5-b1a9-83611324b087 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:25:53.076951 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 843/1673] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:25:52 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:25:53.090408 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba9d273-1e89-4a29-9fee-957be8e7c032 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:25:53.092035 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-fba9d273-1e89-4a29-9fee-957be8e7c032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:25:53.180905 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 831/1674] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 24 17:25:52 2022] POST /identity/v3/projects => generated 383 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:25:53.181986 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af396d0-6c71-45e5-b1a9-83611324b087 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:25:53.232483 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba9d273-1e89-4a29-9fee-957be8e7c032 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:25:53.327992 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:25:53.341805 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af396d0-6c71-45e5-b1a9-83611324b087 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:25:53.358892 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36a6c5a-249c-4bdd-b201-2564f32852c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:25:53.359945 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36a6c5a-249c-4bdd-b201-2564f32852c7 admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:53.360228 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36a6c5a-249c-4bdd-b201-2564f32852c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:53.360502 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36a6c5a-249c-4bdd-b201-2564f32852c7 admin admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:53.361159 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36a6c5a-249c-4bdd-b201-2564f32852c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:25:53.366651 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36a6c5a-249c-4bdd-b201-2564f32852c7 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:25:53.409646 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8635d634-3448-4869-99a2-718ab728ef10 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be06246d47584b529150ceb98a5cfd95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:25:53.410640 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8635d634-3448-4869-99a2-718ab728ef10 service cinder] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:53.411356 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8635d634-3448-4869-99a2-718ab728ef10 service cinder] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:53.411356 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8635d634-3448-4869-99a2-718ab728ef10 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:53.517395 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 832/1675] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 24 17:25:53 2022] POST /identity/v3/users => generated 292 bytes in 336 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:25:53.521115 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36a6c5a-249c-4bdd-b201-2564f32852c7 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:25:53.550519 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36a6c5a-249c-4bdd-b201-2564f32852c7 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:25:53.564442 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d0508d-140a-481d-b94d-b02cb82ffcca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:25:53.565381 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d0508d-140a-481d-b94d-b02cb82ffcca admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:53.569616 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d0508d-140a-481d-b94d-b02cb82ffcca admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:53.569616 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d0508d-140a-481d-b94d-b02cb82ffcca admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:53.569616 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d0508d-140a-481d-b94d-b02cb82ffcca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:25:53.569616 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d0508d-140a-481d-b94d-b02cb82ffcca admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:25:53.584623 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 833/1676] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:25:53 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:25:53.585588 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8635d634-3448-4869-99a2-718ab728ef10 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:25:53.587605 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8635d634-3448-4869-99a2-718ab728ef10 service cinder] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:25:53.589336 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d0508d-140a-481d-b94d-b02cb82ffcca admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:25:53.614997 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d0508d-140a-481d-b94d-b02cb82ffcca admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:25:53.632218 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3632c2e0-e34b-4064-9c0d-1d97530fb4e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:25:53.632858 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3632c2e0-e34b-4064-9c0d-1d97530fb4e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:53.632961 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 844/1677] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:25:53 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 543 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:25:53.633130 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3632c2e0-e34b-4064-9c0d-1d97530fb4e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:53.633558 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3632c2e0-e34b-4064-9c0d-1d97530fb4e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:53.634647 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3632c2e0-e34b-4064-9c0d-1d97530fb4e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:25:53.636575 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3632c2e0-e34b-4064-9c0d-1d97530fb4e7 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:25:53.668932 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 834/1678] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:25:53 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:25:53.673386 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8635d634-3448-4869-99a2-718ab728ef10 service cinder] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:25:53.717620 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8635d634-3448-4869-99a2-718ab728ef10 service cinder] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:25:53.736244 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f7bc85-8a9a-48b7-be39-956cc7812dd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:25:53.736980 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f7bc85-8a9a-48b7-be39-956cc7812dd1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:53.737368 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f7bc85-8a9a-48b7-be39-956cc7812dd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:53.737655 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f7bc85-8a9a-48b7-be39-956cc7812dd1 admin admin] PATH_INFO: `/v3/projects/a4f0f04f254241e0b0944fdfa80343dc/users/ee541f3c0e804adc85104ae3e429a6c5/roles/60dc7e16aee746d5a3e8aa27d7d8525c` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:53.772887 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f7bc85-8a9a-48b7-be39-956cc7812dd1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4f0f04f254241e0b0944fdfa80343dc, user_id=ee541f3c0e804adc85104ae3e429a6c5, role_id=60dc7e16aee746d5a3e8aa27d7d8525c)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:25:53.776234 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f7bc85-8a9a-48b7-be39-956cc7812dd1 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:25:53.807672 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:25:53.908058 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 845/1679] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 24 17:25:53 2022] PUT /identity/v3/projects/a4f0f04f254241e0b0944fdfa80343dc/users/ee541f3c0e804adc85104ae3e429a6c5/roles/60dc7e16aee746d5a3e8aa27d7d8525c => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:25:53.920828 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fde80f-28fb-48a0-880b-81b8bde0984e None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:53.921345 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fde80f-28fb-48a0-880b-81b8bde0984e None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:53.921417 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fde80f-28fb-48a0-880b-81b8bde0984e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:53.978555 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-d6fde80f-28fb-48a0-880b-81b8bde0984e None None] MFA Rules not processed for user `ee541f3c0e804adc85104ae3e429a6c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:25:53.979798 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-d6fde80f-28fb-48a0-880b-81b8bde0984e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:25:54.122150 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f7bc85-8a9a-48b7-be39-956cc7812dd1 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:25:54.134512 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 835/1680] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:25:53 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:25:54.263805 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f7bc85-8a9a-48b7-be39-956cc7812dd1 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:25:54.329592 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1475bfd5-178a-47e5-91a7-5fb699c80568 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:25:54.329592 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1475bfd5-178a-47e5-91a7-5fb699c80568 admin admin] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:54.329592 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1475bfd5-178a-47e5-91a7-5fb699c80568 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:54.329592 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1475bfd5-178a-47e5-91a7-5fb699c80568 admin admin] PATH_INFO: `/v3/projects` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:54.330030 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1475bfd5-178a-47e5-91a7-5fb699c80568 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:25:54.330030 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1475bfd5-178a-47e5-91a7-5fb699c80568 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:25:54.429310 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243c31c6-0f38-4886-bfbc-a3c9931450fd None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:54.429310 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243c31c6-0f38-4886-bfbc-a3c9931450fd None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:54.430131 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243c31c6-0f38-4886-bfbc-a3c9931450fd None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:54.438229 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 836/1681] 10.222.0.18 () {54 vars in 921 bytes} [Wed Aug 24 17:25:54 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:25:54.466013 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28662d97-96f4-4653-809a-3d15b8d260a9 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:54.466362 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28662d97-96f4-4653-809a-3d15b8d260a9 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:54.466641 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28662d97-96f4-4653-809a-3d15b8d260a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:54.467340 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 846/1682] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 24 17:25:54 2022] POST /identity/v3/projects => generated 383 bytes in 346 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:25:54.471637 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1475bfd5-178a-47e5-91a7-5fb699c80568 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:25:54.563504 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-28662d97-96f4-4653-809a-3d15b8d260a9 None None] MFA Rules not processed for user `03fc77c6107e430f90f7b6f7cfd4ec02`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:25:54.565519 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-28662d97-96f4-4653-809a-3d15b8d260a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:25:54.693722 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1475bfd5-178a-47e5-91a7-5fb699c80568 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:25:54.704695 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4fdcea-1452-4aab-b5eb-6a23c40be60b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:25:54.705458 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4fdcea-1452-4aab-b5eb-6a23c40be60b admin admin] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:54.705843 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4fdcea-1452-4aab-b5eb-6a23c40be60b admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:54.707201 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4fdcea-1452-4aab-b5eb-6a23c40be60b admin admin] PATH_INFO: `/v3/users` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:54.708407 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4fdcea-1452-4aab-b5eb-6a23c40be60b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:25:54.713597 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4fdcea-1452-4aab-b5eb-6a23c40be60b admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:25:54.771757 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 837/1683] 10.222.0.18 () {58 vars in 1018 bytes} [Wed Aug 24 17:25:54 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 309 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:25:54.801213 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:25:54.841213 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 847/1684] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 24 17:25:54 2022] POST /identity/v3/users => generated 298 bytes in 369 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:25:54.846437 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28662d97-96f4-4653-809a-3d15b8d260a9 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:25:54.864916 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28662d97-96f4-4653-809a-3d15b8d260a9 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:25:54.885175 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280bd062-d2e9-4b9c-9670-00a3edc17e65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:25:54.885823 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280bd062-d2e9-4b9c-9670-00a3edc17e65 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:54.886118 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280bd062-d2e9-4b9c-9670-00a3edc17e65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:54.886381 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280bd062-d2e9-4b9c-9670-00a3edc17e65 admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:54.887694 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280bd062-d2e9-4b9c-9670-00a3edc17e65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:25:54.889411 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280bd062-d2e9-4b9c-9670-00a3edc17e65 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:25:54.922500 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 838/1685] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:25:54 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:25:54.926631 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4fdcea-1452-4aab-b5eb-6a23c40be60b admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:25:54.949248 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4fdcea-1452-4aab-b5eb-6a23c40be60b admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:25:54.964195 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e235c5c-2cc9-491e-a84c-d30a1b1d4824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:25:54.979381 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e235c5c-2cc9-491e-a84c-d30a1b1d4824 admin admin] REQUEST_METHOD: `PUT` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:54.979381 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e235c5c-2cc9-491e-a84c-d30a1b1d4824 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:54.979381 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e235c5c-2cc9-491e-a84c-d30a1b1d4824 admin admin] PATH_INFO: `/v3/projects/4209c16dbbda405ca4cdca13c1427caa/users/71a27715f4ae4d24bbbe4c37a4b6eb3c/roles/c5346bc54abf445f9900edf94264c850` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:55.029483 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e235c5c-2cc9-491e-a84c-d30a1b1d4824 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4209c16dbbda405ca4cdca13c1427caa, user_id=71a27715f4ae4d24bbbe4c37a4b6eb3c, role_id=c5346bc54abf445f9900edf94264c850)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:25:55.033044 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e235c5c-2cc9-491e-a84c-d30a1b1d4824 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:25:55.111369 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa555b9-fea3-4f58-b6f4-7f67c62ccdb2 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:55.111654 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa555b9-fea3-4f58-b6f4-7f67c62ccdb2 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:55.111932 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa555b9-fea3-4f58-b6f4-7f67c62ccdb2 None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:55.113752 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 839/1686] 10.222.0.18 () {54 vars in 961 bytes} [Wed Aug 24 17:25:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 17:25:55.120478 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338ab482-0a69-4c8c-9c28-0e6a6b971242 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:55.120769 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338ab482-0a69-4c8c-9c28-0e6a6b971242 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:55.121047 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338ab482-0a69-4c8c-9c28-0e6a6b971242 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:55.160717 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-338ab482-0a69-4c8c-9c28-0e6a6b971242 None None] MFA Rules not processed for user `03fc77c6107e430f90f7b6f7cfd4ec02`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:25:55.161986 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-338ab482-0a69-4c8c-9c28-0e6a6b971242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:25:55.194010 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 848/1687] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 24 17:25:54 2022] PUT /identity/v3/projects/4209c16dbbda405ca4cdca13c1427caa/users/71a27715f4ae4d24bbbe4c37a4b6eb3c/roles/c5346bc54abf445f9900edf94264c850 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:25:55.196066 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 840/1688] 10.222.0.18 () {58 vars in 1058 bytes} [Wed Aug 24 17:25:55 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Aug 24 17:25:55.206307 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e235c5c-2cc9-491e-a84c-d30a1b1d4824 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:25:55.211934 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338ab482-0a69-4c8c-9c28-0e6a6b971242 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:25:55.245650 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338ab482-0a69-4c8c-9c28-0e6a6b971242 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:25:55.314204 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:25:55.410088 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e235c5c-2cc9-491e-a84c-d30a1b1d4824 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:25:55.424126 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009b9cfb-be11-4464-aec9-5a00bc135d60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:25:55.425047 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009b9cfb-be11-4464-aec9-5a00bc135d60 admin admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:55.425508 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009b9cfb-be11-4464-aec9-5a00bc135d60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:55.425932 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009b9cfb-be11-4464-aec9-5a00bc135d60 admin admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:55.426743 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009b9cfb-be11-4464-aec9-5a00bc135d60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:25:55.434486 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009b9cfb-be11-4464-aec9-5a00bc135d60 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:25:55.438307 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02468cb-87a7-4ebd-b63e-974edba16341 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03fc77c6107e430f90f7b6f7cfd4ec02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:25:55.439033 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02468cb-87a7-4ebd-b63e-974edba16341 service placement] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:55.439194 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02468cb-87a7-4ebd-b63e-974edba16341 service placement] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:55.439445 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02468cb-87a7-4ebd-b63e-974edba16341 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:55.458491 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02468cb-87a7-4ebd-b63e-974edba16341 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:25:55.460146 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02468cb-87a7-4ebd-b63e-974edba16341 service placement] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:25:55.465192 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 849/1689] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:25:55 2022] GET /identity/v3/roles => generated 1420 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:25:55.467388 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822435ea-6b63-420d-8ca2-789f7f04662e None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:55.468149 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822435ea-6b63-420d-8ca2-789f7f04662e None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:55.468591 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822435ea-6b63-420d-8ca2-789f7f04662e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:55.486717 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 841/1690] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:25:55 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:25:55.487675 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02468cb-87a7-4ebd-b63e-974edba16341 service placement] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:25:55.509763 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02468cb-87a7-4ebd-b63e-974edba16341 service placement] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:25:55.525042 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf84a13f-3d32-4321-af0d-a1b68d2933d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:25:55.525657 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf84a13f-3d32-4321-af0d-a1b68d2933d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:55.525944 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf84a13f-3d32-4321-af0d-a1b68d2933d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:55.526216 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf84a13f-3d32-4321-af0d-a1b68d2933d7 admin admin] PATH_INFO: `/v3/users/fda5136183484d6496da30da3d55fe4d` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:55.537873 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf84a13f-3d32-4321-af0d-a1b68d2933d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fda5136183484d6496da30da3d55fe4d)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:25:55.537873 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf84a13f-3d32-4321-af0d-a1b68d2933d7 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:25:55.552606 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-822435ea-6b63-420d-8ca2-789f7f04662e None None] MFA Rules not processed for user `ef9e96b709b345c08a4ea10f14211668`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:25:55.554487 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-822435ea-6b63-420d-8ca2-789f7f04662e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:25:55.746958 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 850/1691] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:25:55 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:25:55.746958 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822435ea-6b63-420d-8ca2-789f7f04662e None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:25:55.775200 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822435ea-6b63-420d-8ca2-789f7f04662e None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:25:55.780948 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c30117-182c-4945-9973-47164c89ad5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:25:55.781944 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c30117-182c-4945-9973-47164c89ad5a admin admin] REQUEST_METHOD: `PUT` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:55.782486 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c30117-182c-4945-9973-47164c89ad5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:55.783614 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c30117-182c-4945-9973-47164c89ad5a admin admin] PATH_INFO: `/v3/projects/4209c16dbbda405ca4cdca13c1427caa/users/71a27715f4ae4d24bbbe4c37a4b6eb3c/roles/c5346bc54abf445f9900edf94264c850` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:55.801100 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c30117-182c-4945-9973-47164c89ad5a admin admin] RBAC: Authorizing `identity:create_grant(project_id=4209c16dbbda405ca4cdca13c1427caa, user_id=71a27715f4ae4d24bbbe4c37a4b6eb3c, role_id=c5346bc54abf445f9900edf94264c850)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:25:55.829727 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c30117-182c-4945-9973-47164c89ad5a admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:25:55.860057 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 851/1692] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 24 17:25:55 2022] PUT /identity/v3/projects/4209c16dbbda405ca4cdca13c1427caa/users/71a27715f4ae4d24bbbe4c37a4b6eb3c/roles/c5346bc54abf445f9900edf94264c850 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:25:55.861037 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c30117-182c-4945-9973-47164c89ad5a admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:25:55.864415 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-44c30117-182c-4945-9973-47164c89ad5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:25:55.944105 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-bf84a13f-3d32-4321-af0d-a1b68d2933d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fda5136183484d6496da30da3d55fe4d'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:25:56.029996 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-bf84a13f-3d32-4321-af0d-a1b68d2933d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fda5136183484d6496da30da3d55fe4d'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:25:56.030730 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-bf84a13f-3d32-4321-af0d-a1b68d2933d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fda5136183484d6496da30da3d55fe4d'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:25:56.038339 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c30117-182c-4945-9973-47164c89ad5a admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:25:56.041551 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-bf84a13f-3d32-4321-af0d-a1b68d2933d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fda5136183484d6496da30da3d55fe4d'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:25:56.046252 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4d47ed-c8eb-411e-ad8c-68196b8a6da3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557aca9255cb4bbb8af52e796c5c5177', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:25:56.046973 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4d47ed-c8eb-411e-ad8c-68196b8a6da3 service nova] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:56.047261 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4d47ed-c8eb-411e-ad8c-68196b8a6da3 service nova] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:56.047365 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-bf84a13f-3d32-4321-af0d-a1b68d2933d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fda5136183484d6496da30da3d55fe4d'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:25:56.047543 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4d47ed-c8eb-411e-ad8c-68196b8a6da3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:56.049606 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-6f4d47ed-c8eb-411e-ad8c-68196b8a6da3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:25:56.106550 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 842/1693] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 24 17:25:55 2022] DELETE /identity/v3/users/fda5136183484d6496da30da3d55fe4d => generated 0 bytes in 619 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:25:56.110582 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b92de5e-1ad7-4978-b719-3c7721732b96 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:56.110929 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b92de5e-1ad7-4978-b719-3c7721732b96 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:56.111344 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b92de5e-1ad7-4978-b719-3c7721732b96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:56.113981 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:25:56.163701 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4d47ed-c8eb-411e-ad8c-68196b8a6da3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:25:56.165300 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4d47ed-c8eb-411e-ad8c-68196b8a6da3 service nova] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:25:56.185758 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 852/1694] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:25:55 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:25:56.187209 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd8b9ae-9ee7-48e5-a3e3-96dd7e8f0e2c None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:56.187879 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd8b9ae-9ee7-48e5-a3e3-96dd7e8f0e2c None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:56.188752 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd8b9ae-9ee7-48e5-a3e3-96dd7e8f0e2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:56.202540 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-8b92de5e-1ad7-4978-b719-3c7721732b96 None None] MFA Rules not processed for user `71a27715f4ae4d24bbbe4c37a4b6eb3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:25:56.203846 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-8b92de5e-1ad7-4978-b719-3c7721732b96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:25:56.219408 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-cdd8b9ae-9ee7-48e5-a3e3-96dd7e8f0e2c None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:25:56.220720 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-cdd8b9ae-9ee7-48e5-a3e3-96dd7e8f0e2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:25:56.328009 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 853/1695] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:25:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:25:56.373435 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 843/1696] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:25:56 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:25:56.384942 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd8b9ae-9ee7-48e5-a3e3-96dd7e8f0e2c None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:25:56.386481 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-cdd8b9ae-9ee7-48e5-a3e3-96dd7e8f0e2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:25:56.417708 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b92de5e-1ad7-4978-b719-3c7721732b96 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:25:56.419777 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-8b92de5e-1ad7-4978-b719-3c7721732b96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:25:56.435861 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:25:56.516646 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b92de5e-1ad7-4978-b719-3c7721732b96 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:25:56.524894 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f2cf25-4b37-44cf-9a0b-abf2f00ad929 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e9ed4d47aef4140bfb2d3bb798cc559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:25:56.525881 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f2cf25-4b37-44cf-9a0b-abf2f00ad929 service neutron] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:56.526170 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f2cf25-4b37-44cf-9a0b-abf2f00ad929 service neutron] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:56.526462 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f2cf25-4b37-44cf-9a0b-abf2f00ad929 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:56.548008 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd8b9ae-9ee7-48e5-a3e3-96dd7e8f0e2c None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:25:56.550653 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f2cf25-4b37-44cf-9a0b-abf2f00ad929 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:25:56.557322 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f2cf25-4b37-44cf-9a0b-abf2f00ad929 service neutron] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:25:56.561974 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259fefa4-7584-44b2-a864-b16b7e1f0175 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be06246d47584b529150ceb98a5cfd95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:25:56.562946 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259fefa4-7584-44b2-a864-b16b7e1f0175 service cinder] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:56.563638 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259fefa4-7584-44b2-a864-b16b7e1f0175 service cinder] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:56.564104 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259fefa4-7584-44b2-a864-b16b7e1f0175 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:56.566821 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:25:56.585257 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 844/1697] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:25:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:25:56.588775 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259fefa4-7584-44b2-a864-b16b7e1f0175 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:25:56.593756 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259fefa4-7584-44b2-a864-b16b7e1f0175 service cinder] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:25:56.621194 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 854/1698] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:25:56 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:25:56.823308 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f2cf25-4b37-44cf-9a0b-abf2f00ad929 service neutron] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:25:56.840905 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f2cf25-4b37-44cf-9a0b-abf2f00ad929 service neutron] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:25:56.852152 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28546a08-f8b8-4657-ad95-4747a5660fb5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be06246d47584b529150ceb98a5cfd95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:25:56.852993 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28546a08-f8b8-4657-ad95-4747a5660fb5 service cinder] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:56.853443 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28546a08-f8b8-4657-ad95-4747a5660fb5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:56.853721 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28546a08-f8b8-4657-ad95-4747a5660fb5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:56.855625 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-28546a08-f8b8-4657-ad95-4747a5660fb5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:25:57.017665 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28546a08-f8b8-4657-ad95-4747a5660fb5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:25:57.019390 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28546a08-f8b8-4657-ad95-4747a5660fb5 service cinder] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:25:57.035283 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 845/1699] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:25:56 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:25:57.466485 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5330e8-d60c-4e76-acbb-5b3e745f69a0 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:57.467041 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5330e8-d60c-4e76-acbb-5b3e745f69a0 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:57.467455 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5330e8-d60c-4e76-acbb-5b3e745f69a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:57.532798 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-ab5330e8-d60c-4e76-acbb-5b3e745f69a0 None None] MFA Rules not processed for user `bb2bc3ba87a549018d50fe3c9c6720d0`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:25:57.535020 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-ab5330e8-d60c-4e76-acbb-5b3e745f69a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:25:57.698415 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369df942-dca4-4be2-8a3e-7eec46535927 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:57.698706 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369df942-dca4-4be2-8a3e-7eec46535927 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:57.699009 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369df942-dca4-4be2-8a3e-7eec46535927 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:57.716068 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 855/1700] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:25:57 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:25:57.728014 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ebac1c-bf2d-4fc8-9385-c76ab69fc81e None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:57.728607 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ebac1c-bf2d-4fc8-9385-c76ab69fc81e None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:57.729092 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ebac1c-bf2d-4fc8-9385-c76ab69fc81e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:57.766036 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-a8ebac1c-bf2d-4fc8-9385-c76ab69fc81e None None] MFA Rules not processed for user `bb2bc3ba87a549018d50fe3c9c6720d0`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:25:57.767540 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-a8ebac1c-bf2d-4fc8-9385-c76ab69fc81e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:25:57.784734 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 856/1701] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:25:57 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:25:57.796054 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ebac1c-bf2d-4fc8-9385-c76ab69fc81e None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:25:57.797605 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-a8ebac1c-bf2d-4fc8-9385-c76ab69fc81e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:25:57.798528 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-369df942-dca4-4be2-8a3e-7eec46535927 None None] MFA Rules not processed for user `0696e6f707274727b57a9a7493c47664`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:25:57.799986 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-369df942-dca4-4be2-8a3e-7eec46535927 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:25:57.899368 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 846/1702] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:25:57 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:25:57.912404 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9308673d-a1c7-4500-9b1e-3ad92e0ca3fe None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:57.914862 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9308673d-a1c7-4500-9b1e-3ad92e0ca3fe None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:57.918372 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9308673d-a1c7-4500-9b1e-3ad92e0ca3fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:57.963830 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-9308673d-a1c7-4500-9b1e-3ad92e0ca3fe None None] MFA Rules not processed for user `0696e6f707274727b57a9a7493c47664`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:25:57.965076 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-9308673d-a1c7-4500-9b1e-3ad92e0ca3fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:25:57.990958 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ebac1c-bf2d-4fc8-9385-c76ab69fc81e None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:25:57.991159 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 847/1703] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:25:57 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:25:58.004758 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9308673d-a1c7-4500-9b1e-3ad92e0ca3fe None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:25:58.004936 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-9308673d-a1c7-4500-9b1e-3ad92e0ca3fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:25:58.008327 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0b99b8-bf15-412f-8935-5f072bea461d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557aca9255cb4bbb8af52e796c5c5177', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:25:58.008961 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0b99b8-bf15-412f-8935-5f072bea461d service nova] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:58.009240 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0b99b8-bf15-412f-8935-5f072bea461d service nova] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:58.009511 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0b99b8-bf15-412f-8935-5f072bea461d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:58.030265 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0b99b8-bf15-412f-8935-5f072bea461d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:25:58.031976 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0b99b8-bf15-412f-8935-5f072bea461d service nova] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:25:58.037527 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9308673d-a1c7-4500-9b1e-3ad92e0ca3fe None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:25:58.048933 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 857/1704] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:25:57 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:25:58.066407 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d989709f-8311-49b7-9e23-a848ec1b649b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557aca9255cb4bbb8af52e796c5c5177', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:25:58.083270 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d989709f-8311-49b7-9e23-a848ec1b649b service nova] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:58.083548 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d989709f-8311-49b7-9e23-a848ec1b649b service nova] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:58.083827 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d989709f-8311-49b7-9e23-a848ec1b649b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:58.117908 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d989709f-8311-49b7-9e23-a848ec1b649b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:25:58.129616 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d989709f-8311-49b7-9e23-a848ec1b649b service nova] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:25:58.145344 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 848/1705] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:25:58 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:25:58.256393 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e6cab5-ed91-4cb6-8e36-a69261ac457b None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:58.256685 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e6cab5-ed91-4cb6-8e36-a69261ac457b None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:58.257003 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e6cab5-ed91-4cb6-8e36-a69261ac457b None None] PATH_INFO: `/` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:58.258735 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 858/1706] 10.222.0.18 () {54 vars in 921 bytes} [Wed Aug 24 17:25:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:25:58.278781 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1331df39-0007-4afe-a476-3d6de8a01950 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:58.278781 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1331df39-0007-4afe-a476-3d6de8a01950 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:58.279019 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1331df39-0007-4afe-a476-3d6de8a01950 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:58.332063 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-1331df39-0007-4afe-a476-3d6de8a01950 None None] MFA Rules not processed for user `03fc77c6107e430f90f7b6f7cfd4ec02`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:25:58.336937 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-1331df39-0007-4afe-a476-3d6de8a01950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:25:58.373575 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:25:58.597218 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 849/1707] 10.222.0.18 () {58 vars in 1018 bytes} [Wed Aug 24 17:25:58 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 321 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:25:58.770676 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e6cab5-ed91-4cb6-8e36-a69261ac457b None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:25:58.772569 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-36e6cab5-ed91-4cb6-8e36-a69261ac457b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:25:58.797489 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1331df39-0007-4afe-a476-3d6de8a01950 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:25:58.799186 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-1331df39-0007-4afe-a476-3d6de8a01950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:25:58.838945 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e6cab5-ed91-4cb6-8e36-a69261ac457b None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:25:58.840519 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1331df39-0007-4afe-a476-3d6de8a01950 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:25:58.857524 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e937c3b-1fd4-40b0-8860-d25b42634e92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:25:58.858621 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e937c3b-1fd4-40b0-8860-d25b42634e92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:58.860959 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e937c3b-1fd4-40b0-8860-d25b42634e92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:58.861523 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e937c3b-1fd4-40b0-8860-d25b42634e92 admin admin] PATH_INFO: `/v3/users/0be478bd2b6046cd93c368f531be477c` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:58.866200 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54831fde-c056-460c-a9cb-2a351101e037 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03fc77c6107e430f90f7b6f7cfd4ec02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:25:58.866542 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e937c3b-1fd4-40b0-8860-d25b42634e92 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0be478bd2b6046cd93c368f531be477c)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:25:58.867177 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54831fde-c056-460c-a9cb-2a351101e037 service placement] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:58.867547 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54831fde-c056-460c-a9cb-2a351101e037 service placement] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:58.867892 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54831fde-c056-460c-a9cb-2a351101e037 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:58.869792 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e937c3b-1fd4-40b0-8860-d25b42634e92 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:25:58.900798 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54831fde-c056-460c-a9cb-2a351101e037 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:25:58.902463 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54831fde-c056-460c-a9cb-2a351101e037 service placement] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:25:58.931079 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 850/1708] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:25:58 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:25:59.033421 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:25:59.095249 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5f514f-c8b2-4fb7-970c-f1ab90400df1 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:59.095776 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5f514f-c8b2-4fb7-970c-f1ab90400df1 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:59.096046 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5f514f-c8b2-4fb7-970c-f1ab90400df1 None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:59.099265 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 851/1709] 10.222.0.18 () {54 vars in 912 bytes} [Wed Aug 24 17:25:59 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:25:59.110583 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee2be1b-bc12-4105-8327-78d2982cdb52 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:59.111439 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee2be1b-bc12-4105-8327-78d2982cdb52 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:59.112646 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee2be1b-bc12-4105-8327-78d2982cdb52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:59.141423 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-6e937c3b-1fd4-40b0-8860-d25b42634e92 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0be478bd2b6046cd93c368f531be477c'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:25:59.170421 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-cee2be1b-bc12-4105-8327-78d2982cdb52 None None] MFA Rules not processed for user `0e9ed4d47aef4140bfb2d3bb798cc559`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:25:59.172516 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-cee2be1b-bc12-4105-8327-78d2982cdb52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:25:59.180396 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-6e937c3b-1fd4-40b0-8860-d25b42634e92 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0be478bd2b6046cd93c368f531be477c'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:25:59.191425 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 852/1710] 10.222.0.18 () {58 vars in 1009 bytes} [Wed Aug 24 17:25:59 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:25:59.258831 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee2be1b-bc12-4105-8327-78d2982cdb52 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:25:59.263202 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-cee2be1b-bc12-4105-8327-78d2982cdb52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:25:59.286307 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee2be1b-bc12-4105-8327-78d2982cdb52 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:25:59.299556 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe6ac6b-ee66-4b15-b4a0-b57232143186 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e9ed4d47aef4140bfb2d3bb798cc559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:25:59.300589 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe6ac6b-ee66-4b15-b4a0-b57232143186 service neutron] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:59.300589 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe6ac6b-ee66-4b15-b4a0-b57232143186 service neutron] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:59.300801 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe6ac6b-ee66-4b15-b4a0-b57232143186 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:59.310217 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-6e937c3b-1fd4-40b0-8860-d25b42634e92 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0be478bd2b6046cd93c368f531be477c'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:25:59.311122 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-6e937c3b-1fd4-40b0-8860-d25b42634e92 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0be478bd2b6046cd93c368f531be477c'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:25:59.311504 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe6ac6b-ee66-4b15-b4a0-b57232143186 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:25:59.315592 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe6ac6b-ee66-4b15-b4a0-b57232143186 service neutron] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:25:59.320049 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-6e937c3b-1fd4-40b0-8860-d25b42634e92 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0be478bd2b6046cd93c368f531be477c'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:25:59.337683 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 859/1711] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 24 17:25:58 2022] DELETE /identity/v3/users/0be478bd2b6046cd93c368f531be477c => generated 0 bytes in 567 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:25:59.341740 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 853/1712] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:25:59 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:25:59.451105 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec46cff-73ae-49d5-be7e-a5170fbab3a0 None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:59.451397 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec46cff-73ae-49d5-be7e-a5170fbab3a0 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:59.451674 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec46cff-73ae-49d5-be7e-a5170fbab3a0 None None] PATH_INFO: `/` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:59.453567 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 860/1713] 10.222.0.18 () {54 vars in 912 bytes} [Wed Aug 24 17:25:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:25:59.459491 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539ad1ae-5fbf-42f1-9ee1-d8f268c86367 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:59.459848 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539ad1ae-5fbf-42f1-9ee1-d8f268c86367 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:59.460239 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539ad1ae-5fbf-42f1-9ee1-d8f268c86367 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:59.500577 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-539ad1ae-5fbf-42f1-9ee1-d8f268c86367 None None] MFA Rules not processed for user `0e9ed4d47aef4140bfb2d3bb798cc559`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:25:59.501247 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-539ad1ae-5fbf-42f1-9ee1-d8f268c86367 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:25:59.660985 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 854/1714] 10.222.0.18 () {58 vars in 1009 bytes} [Wed Aug 24 17:25:59 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:25:59.818661 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec46cff-73ae-49d5-be7e-a5170fbab3a0 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:25:59.821148 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-5ec46cff-73ae-49d5-be7e-a5170fbab3a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:25:59.849460 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec46cff-73ae-49d5-be7e-a5170fbab3a0 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:25:59.859831 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a1fddb-15df-4b2e-b76a-7f34c2277567 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e9ed4d47aef4140bfb2d3bb798cc559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:25:59.863161 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a1fddb-15df-4b2e-b76a-7f34c2277567 service neutron] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:25:59.863448 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a1fddb-15df-4b2e-b76a-7f34c2277567 service neutron] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:25:59.864976 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a1fddb-15df-4b2e-b76a-7f34c2277567 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:25:59.894446 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a1fddb-15df-4b2e-b76a-7f34c2277567 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:25:59.896401 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a1fddb-15df-4b2e-b76a-7f34c2277567 service neutron] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:25:59.915622 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 861/1715] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:25:59 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:26:00.266786 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539ad1ae-5fbf-42f1-9ee1-d8f268c86367 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:00.268845 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-539ad1ae-5fbf-42f1-9ee1-d8f268c86367 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:26:00.292004 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a1fddb-15df-4b2e-b76a-7f34c2277567 service neutron] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:00.295599 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-21a1fddb-15df-4b2e-b76a-7f34c2277567 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:26:00.492682 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a1fddb-15df-4b2e-b76a-7f34c2277567 service neutron] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:00.500661 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415075c6-d423-4b42-b727-c6cb52a4854e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:00.502389 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415075c6-d423-4b42-b727-c6cb52a4854e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:00.502970 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415075c6-d423-4b42-b727-c6cb52a4854e admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:00.503566 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415075c6-d423-4b42-b727-c6cb52a4854e admin admin] PATH_INFO: `/v3/projects/d88bff95d6084fc3a2623ad6e4a58d68` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:00.506288 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415075c6-d423-4b42-b727-c6cb52a4854e admin admin] RBAC: Authorizing `identity:delete_project(project_id=d88bff95d6084fc3a2623ad6e4a58d68)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:00.511540 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415075c6-d423-4b42-b727-c6cb52a4854e admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:00.568250 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539ad1ae-5fbf-42f1-9ee1-d8f268c86367 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:00.592066 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcaf22d4-8c33-4516-9531-c022e0ec14ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:00.592830 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaf22d4-8c33-4516-9531-c022e0ec14ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:00.593112 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaf22d4-8c33-4516-9531-c022e0ec14ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:00.593390 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaf22d4-8c33-4516-9531-c022e0ec14ef admin admin] PATH_INFO: `/v3/projects/5239fb586bd043c394710fa651720dbc` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:00.595691 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcaf22d4-8c33-4516-9531-c022e0ec14ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=5239fb586bd043c394710fa651720dbc)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:00.597692 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcaf22d4-8c33-4516-9531-c022e0ec14ef admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:00.653646 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-415075c6-d423-4b42-b727-c6cb52a4854e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd88bff95d6084fc3a2623ad6e4a58d68', 'request_id': 'req-415075c6-d423-4b42-b727-c6cb52a4854e'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:00.677937 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:26:00.756170 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-415075c6-d423-4b42-b727-c6cb52a4854e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd88bff95d6084fc3a2623ad6e4a58d68', 'request_id': 'req-415075c6-d423-4b42-b727-c6cb52a4854e'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:00.768652 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-415075c6-d423-4b42-b727-c6cb52a4854e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd88bff95d6084fc3a2623ad6e4a58d68', 'request_id': 'req-415075c6-d423-4b42-b727-c6cb52a4854e'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:00.792564 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-dcaf22d4-8c33-4516-9531-c022e0ec14ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5239fb586bd043c394710fa651720dbc', 'request_id': 'req-dcaf22d4-8c33-4516-9531-c022e0ec14ef'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:00.805327 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-415075c6-d423-4b42-b727-c6cb52a4854e admin admin] The token cache is being invalidate because project d88bff95d6084fc3a2623ad6e4a58d68 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81731) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:26:00.805712 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-415075c6-d423-4b42-b727-c6cb52a4854e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:00.807265 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-415075c6-d423-4b42-b727-c6cb52a4854e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:00.810418 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 862/1716] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 24 17:26:00 2022] DELETE /identity/v3/projects/d88bff95d6084fc3a2623ad6e4a58d68 => generated 0 bytes in 519 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:26:00.814506 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415075c6-d423-4b42-b727-c6cb52a4854e admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:00.821516 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-415075c6-d423-4b42-b727-c6cb52a4854e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:26:00.940989 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-dcaf22d4-8c33-4516-9531-c022e0ec14ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5239fb586bd043c394710fa651720dbc', 'request_id': 'req-dcaf22d4-8c33-4516-9531-c022e0ec14ef'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:00.947346 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:26:00.952011 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-dcaf22d4-8c33-4516-9531-c022e0ec14ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5239fb586bd043c394710fa651720dbc', 'request_id': 'req-dcaf22d4-8c33-4516-9531-c022e0ec14ef'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:00.987629 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-dcaf22d4-8c33-4516-9531-c022e0ec14ef admin admin] The token cache is being invalidate because project 5239fb586bd043c394710fa651720dbc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81732) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:26:00.988093 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-dcaf22d4-8c33-4516-9531-c022e0ec14ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:00.992764 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-dcaf22d4-8c33-4516-9531-c022e0ec14ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:00.992764 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 855/1717] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 24 17:26:00 2022] DELETE /identity/v3/projects/5239fb586bd043c394710fa651720dbc => generated 0 bytes in 727 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:26:00.997677 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcaf22d4-8c33-4516-9531-c022e0ec14ef admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:01.012397 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-dcaf22d4-8c33-4516-9531-c022e0ec14ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:26:01.089471 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415075c6-d423-4b42-b727-c6cb52a4854e admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:01.256293 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcaf22d4-8c33-4516-9531-c022e0ec14ef admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:01.273035 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa65bf0-dd5e-4b99-93d2-2f54b3f102ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:01.274532 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa65bf0-dd5e-4b99-93d2-2f54b3f102ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:01.275005 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa65bf0-dd5e-4b99-93d2-2f54b3f102ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:01.275885 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa65bf0-dd5e-4b99-93d2-2f54b3f102ac admin admin] PATH_INFO: `/v3/users/fffd224b28f9487d8652d4183e4a2d5d` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:01.284811 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa65bf0-dd5e-4b99-93d2-2f54b3f102ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=fffd224b28f9487d8652d4183e4a2d5d)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:01.287638 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa65bf0-dd5e-4b99-93d2-2f54b3f102ac admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:01.329896 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cb7a57-51c4-43b5-ada7-6b132f9a9dbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:01.329896 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cb7a57-51c4-43b5-ada7-6b132f9a9dbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:01.329896 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cb7a57-51c4-43b5-ada7-6b132f9a9dbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:01.331427 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cb7a57-51c4-43b5-ada7-6b132f9a9dbd admin admin] PATH_INFO: `/v3/users/389fce298f2c42cabe4ceca307b38151` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:01.332948 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cb7a57-51c4-43b5-ada7-6b132f9a9dbd admin admin] RBAC: Authorizing `identity:delete_user(user_id=389fce298f2c42cabe4ceca307b38151)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:01.337484 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cb7a57-51c4-43b5-ada7-6b132f9a9dbd admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:01.670263 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-8aa65bf0-dd5e-4b99-93d2-2f54b3f102ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fffd224b28f9487d8652d4183e4a2d5d'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:01.698427 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-e0cb7a57-51c4-43b5-ada7-6b132f9a9dbd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '389fce298f2c42cabe4ceca307b38151'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:01.727063 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-e0cb7a57-51c4-43b5-ada7-6b132f9a9dbd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '389fce298f2c42cabe4ceca307b38151'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:01.758939 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-8aa65bf0-dd5e-4b99-93d2-2f54b3f102ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fffd224b28f9487d8652d4183e4a2d5d'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:01.759891 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-8aa65bf0-dd5e-4b99-93d2-2f54b3f102ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fffd224b28f9487d8652d4183e4a2d5d'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:01.773086 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-8aa65bf0-dd5e-4b99-93d2-2f54b3f102ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fffd224b28f9487d8652d4183e4a2d5d'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:01.773921 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-8aa65bf0-dd5e-4b99-93d2-2f54b3f102ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fffd224b28f9487d8652d4183e4a2d5d'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:01.839606 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 856/1718] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 24 17:26:00 2022] DELETE /identity/v3/users/fffd224b28f9487d8652d4183e4a2d5d => generated 0 bytes in 843 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:26:01.841202 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa65bf0-dd5e-4b99-93d2-2f54b3f102ac admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:01.843762 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-8aa65bf0-dd5e-4b99-93d2-2f54b3f102ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:26:01.844187 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-e0cb7a57-51c4-43b5-ada7-6b132f9a9dbd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '389fce298f2c42cabe4ceca307b38151'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:01.845169 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-e0cb7a57-51c4-43b5-ada7-6b132f9a9dbd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '389fce298f2c42cabe4ceca307b38151'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:01.858373 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-e0cb7a57-51c4-43b5-ada7-6b132f9a9dbd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '389fce298f2c42cabe4ceca307b38151'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:01.901847 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 863/1719] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 24 17:26:00 2022] DELETE /identity/v3/users/389fce298f2c42cabe4ceca307b38151 => generated 0 bytes in 1088 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:26:01.907107 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:26:01.910932 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cb7a57-51c4-43b5-ada7-6b132f9a9dbd admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:01.923061 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-e0cb7a57-51c4-43b5-ada7-6b132f9a9dbd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:26:01.998074 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa65bf0-dd5e-4b99-93d2-2f54b3f102ac admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:02.034468 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cb7a57-51c4-43b5-ada7-6b132f9a9dbd admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:02.047900 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d9d4d2-0da7-4c0f-a5f1-32fcf5017a63 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03fc77c6107e430f90f7b6f7cfd4ec02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:02.048923 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d9d4d2-0da7-4c0f-a5f1-32fcf5017a63 service placement] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:02.049806 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d9d4d2-0da7-4c0f-a5f1-32fcf5017a63 service placement] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:02.050277 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d9d4d2-0da7-4c0f-a5f1-32fcf5017a63 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:02.052908 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-c2d9d4d2-0da7-4c0f-a5f1-32fcf5017a63 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:26:02.076273 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d9d4d2-0da7-4c0f-a5f1-32fcf5017a63 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:02.078646 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d9d4d2-0da7-4c0f-a5f1-32fcf5017a63 service placement] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:02.105049 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 864/1720] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:26:01 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:26:02.108226 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b143246-1359-4905-9047-0461c5799166 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:02.108789 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b143246-1359-4905-9047-0461c5799166 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:02.109999 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b143246-1359-4905-9047-0461c5799166 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:02.144161 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-4b143246-1359-4905-9047-0461c5799166 None None] MFA Rules not processed for user `0696e6f707274727b57a9a7493c47664`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:26:02.145815 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-4b143246-1359-4905-9047-0461c5799166 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:26:02.149733 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b722e5f-7280-4bbb-a7f8-33f27619087e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03fc77c6107e430f90f7b6f7cfd4ec02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:02.150498 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b722e5f-7280-4bbb-a7f8-33f27619087e service placement] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:02.150630 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b722e5f-7280-4bbb-a7f8-33f27619087e service placement] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:02.150938 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b722e5f-7280-4bbb-a7f8-33f27619087e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:02.176571 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b722e5f-7280-4bbb-a7f8-33f27619087e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:02.178982 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b722e5f-7280-4bbb-a7f8-33f27619087e service placement] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:02.201646 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 857/1721] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:26:01 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:26:02.270679 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 865/1722] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:26:02 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:26:02.317106 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b722e5f-7280-4bbb-a7f8-33f27619087e service placement] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:02.318688 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-0b722e5f-7280-4bbb-a7f8-33f27619087e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:26:02.499872 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b722e5f-7280-4bbb-a7f8-33f27619087e service placement] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:02.509139 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba4ec90-b97a-4b0c-a1ce-41b5da83963e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:02.510084 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba4ec90-b97a-4b0c-a1ce-41b5da83963e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:02.516628 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba4ec90-b97a-4b0c-a1ce-41b5da83963e admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:02.517115 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba4ec90-b97a-4b0c-a1ce-41b5da83963e admin admin] PATH_INFO: `/v3/users/ee541f3c0e804adc85104ae3e429a6c5` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:02.519235 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba4ec90-b97a-4b0c-a1ce-41b5da83963e admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee541f3c0e804adc85104ae3e429a6c5)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:02.524006 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba4ec90-b97a-4b0c-a1ce-41b5da83963e admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:02.791667 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-1ba4ec90-b97a-4b0c-a1ce-41b5da83963e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee541f3c0e804adc85104ae3e429a6c5'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:02.895050 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-1ba4ec90-b97a-4b0c-a1ce-41b5da83963e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee541f3c0e804adc85104ae3e429a6c5'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:02.895957 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-1ba4ec90-b97a-4b0c-a1ce-41b5da83963e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee541f3c0e804adc85104ae3e429a6c5'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:02.920904 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-1ba4ec90-b97a-4b0c-a1ce-41b5da83963e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee541f3c0e804adc85104ae3e429a6c5'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:02.922320 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-1ba4ec90-b97a-4b0c-a1ce-41b5da83963e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee541f3c0e804adc85104ae3e429a6c5'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:02.974165 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 858/1723] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 24 17:26:02 2022] DELETE /identity/v3/users/ee541f3c0e804adc85104ae3e429a6c5 => generated 0 bytes in 654 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:26:02.975112 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14894b9-5977-4d37-afa5-2a519cac9717 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:02.975815 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14894b9-5977-4d37-afa5-2a519cac9717 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:02.976130 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14894b9-5977-4d37-afa5-2a519cac9717 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:03.035058 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-f14894b9-5977-4d37-afa5-2a519cac9717 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:26:03.047455 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-f14894b9-5977-4d37-afa5-2a519cac9717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:26:03.228999 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 866/1724] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:26:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:26:03.271023 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba4ec90-b97a-4b0c-a1ce-41b5da83963e admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:03.277829 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-1ba4ec90-b97a-4b0c-a1ce-41b5da83963e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:26:03.373478 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14894b9-5977-4d37-afa5-2a519cac9717 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:03.377390 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:26:03.377620 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-f14894b9-5977-4d37-afa5-2a519cac9717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:26:03.398537 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14894b9-5977-4d37-afa5-2a519cac9717 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:03.448889 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4342bbc5-0276-48b0-b89e-3880d5928037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:03.450843 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4342bbc5-0276-48b0-b89e-3880d5928037 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:03.451159 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4342bbc5-0276-48b0-b89e-3880d5928037 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:03.451901 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4342bbc5-0276-48b0-b89e-3880d5928037 admin admin] PATH_INFO: `/v3/projects/b9fd422cf1b6454cb92194f346a07830` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:03.454788 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4342bbc5-0276-48b0-b89e-3880d5928037 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9fd422cf1b6454cb92194f346a07830)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:03.457644 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4342bbc5-0276-48b0-b89e-3880d5928037 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:03.496806 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba4ec90-b97a-4b0c-a1ce-41b5da83963e admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:03.510673 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b621f824-bea8-444f-8b7d-e6d4c316c659 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e9ed4d47aef4140bfb2d3bb798cc559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:03.511716 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b621f824-bea8-444f-8b7d-e6d4c316c659 service neutron] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:03.516332 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b621f824-bea8-444f-8b7d-e6d4c316c659 service neutron] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:03.516332 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b621f824-bea8-444f-8b7d-e6d4c316c659 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:03.550667 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b621f824-bea8-444f-8b7d-e6d4c316c659 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:03.553788 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b621f824-bea8-444f-8b7d-e6d4c316c659 service neutron] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:03.583844 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 859/1725] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:26:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:26:03.585174 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb10ed6d-1419-43f5-99e6-a0df3dc9604c None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:03.585460 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb10ed6d-1419-43f5-99e6-a0df3dc9604c None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:03.593534 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb10ed6d-1419-43f5-99e6-a0df3dc9604c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:03.680432 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-4342bbc5-0276-48b0-b89e-3880d5928037 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9fd422cf1b6454cb92194f346a07830', 'request_id': 'req-4342bbc5-0276-48b0-b89e-3880d5928037'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:03.686419 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-fb10ed6d-1419-43f5-99e6-a0df3dc9604c None None] MFA Rules not processed for user `c379c845fccd43c08e8699976d506801`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:26:03.687537 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-fb10ed6d-1419-43f5-99e6-a0df3dc9604c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:26:03.798062 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-4342bbc5-0276-48b0-b89e-3880d5928037 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9fd422cf1b6454cb92194f346a07830', 'request_id': 'req-4342bbc5-0276-48b0-b89e-3880d5928037'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:03.808197 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-4342bbc5-0276-48b0-b89e-3880d5928037 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9fd422cf1b6454cb92194f346a07830', 'request_id': 'req-4342bbc5-0276-48b0-b89e-3880d5928037'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:03.846329 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-4342bbc5-0276-48b0-b89e-3880d5928037 admin admin] The token cache is being invalidate because project b9fd422cf1b6454cb92194f346a07830 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81731) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:26:03.847114 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-4342bbc5-0276-48b0-b89e-3880d5928037 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:03.848308 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-4342bbc5-0276-48b0-b89e-3880d5928037 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:03.858094 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 867/1726] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 24 17:26:03 2022] DELETE /identity/v3/projects/b9fd422cf1b6454cb92194f346a07830 => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:26:03.862558 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e70e95f-7bf6-4727-bfb5-d408f8b99af7 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:03.864571 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e70e95f-7bf6-4727-bfb5-d408f8b99af7 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:03.864846 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e70e95f-7bf6-4727-bfb5-d408f8b99af7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:03.880373 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 860/1727] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:26:03 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:26:03.933909 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-9e70e95f-7bf6-4727-bfb5-d408f8b99af7 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:26:03.935287 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-9e70e95f-7bf6-4727-bfb5-d408f8b99af7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:26:04.039244 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b08429-223d-470f-9fdd-1ba82f76610c None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:04.039685 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b08429-223d-470f-9fdd-1ba82f76610c None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:04.039815 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b08429-223d-470f-9fdd-1ba82f76610c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:04.078843 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-b3b08429-223d-470f-9fdd-1ba82f76610c None None] MFA Rules not processed for user `160ce86c89ea449da346d98d20ce2a9e`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:26:04.080186 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-b3b08429-223d-470f-9fdd-1ba82f76610c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:26:04.090472 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 868/1728] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:26:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:26:04.099129 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed805fac-56b6-4bb0-9f58-840fdf7cd13d None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:04.099426 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed805fac-56b6-4bb0-9f58-840fdf7cd13d None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:04.099708 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed805fac-56b6-4bb0-9f58-840fdf7cd13d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:04.121398 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-ed805fac-56b6-4bb0-9f58-840fdf7cd13d None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:26:04.122641 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-ed805fac-56b6-4bb0-9f58-840fdf7cd13d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:26:04.135354 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 869/1729] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:26:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:26:04.136355 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed805fac-56b6-4bb0-9f58-840fdf7cd13d None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:04.139976 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:26:04.147219 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed805fac-56b6-4bb0-9f58-840fdf7cd13d None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:04.154204 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d10303-429f-4e30-ae74-549e4d792d11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:04.154838 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d10303-429f-4e30-ae74-549e4d792d11 admin admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:04.155182 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d10303-429f-4e30-ae74-549e4d792d11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:04.155446 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d10303-429f-4e30-ae74-549e4d792d11 admin admin] PATH_INFO: `/v3/domains` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:04.156245 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d10303-429f-4e30-ae74-549e4d792d11 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:26:04.156550 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d10303-429f-4e30-ae74-549e4d792d11 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:04.158187 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d10303-429f-4e30-ae74-549e4d792d11 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:04.176156 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 870/1730] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 24 17:26:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:26:04.177446 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d10303-429f-4e30-ae74-549e4d792d11 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:04.191328 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d10303-429f-4e30-ae74-549e4d792d11 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:04.196227 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6071defc-03bb-4f0e-af2a-3d1ea0b7b5c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:04.196885 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6071defc-03bb-4f0e-af2a-3d1ea0b7b5c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:04.197129 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6071defc-03bb-4f0e-af2a-3d1ea0b7b5c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:04.197797 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6071defc-03bb-4f0e-af2a-3d1ea0b7b5c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:04.203811 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6071defc-03bb-4f0e-af2a-3d1ea0b7b5c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:04.206416 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6071defc-03bb-4f0e-af2a-3d1ea0b7b5c1 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:04.235887 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 861/1731] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:26:04 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:26:04.339772 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 871/1732] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 24 17:26:04 2022] POST /identity/v3/projects => generated 379 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:26:04.340545 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b08429-223d-470f-9fdd-1ba82f76610c None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:04.356909 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b08429-223d-470f-9fdd-1ba82f76610c None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:04.401369 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6071defc-03bb-4f0e-af2a-3d1ea0b7b5c1 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:04.404245 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-6071defc-03bb-4f0e-af2a-3d1ea0b7b5c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:26:04.535111 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3fb43c-2736-40ee-9b7d-b119a491ae7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:04.542259 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3fb43c-2736-40ee-9b7d-b119a491ae7b admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:04.542259 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3fb43c-2736-40ee-9b7d-b119a491ae7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:04.542259 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3fb43c-2736-40ee-9b7d-b119a491ae7b admin admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:04.542259 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3fb43c-2736-40ee-9b7d-b119a491ae7b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:04.542259 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3fb43c-2736-40ee-9b7d-b119a491ae7b admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:04.577135 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6071defc-03bb-4f0e-af2a-3d1ea0b7b5c1 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:04.583235 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20f0d49-0e6e-4630-9c80-7dfbf02773c6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557aca9255cb4bbb8af52e796c5c5177', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:04.584268 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20f0d49-0e6e-4630-9c80-7dfbf02773c6 service nova] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:04.584792 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20f0d49-0e6e-4630-9c80-7dfbf02773c6 service nova] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:04.585326 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20f0d49-0e6e-4630-9c80-7dfbf02773c6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:04.671811 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 862/1733] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 24 17:26:04 2022] POST /identity/v3/users => generated 290 bytes in 332 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:26:04.675546 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3fb43c-2736-40ee-9b7d-b119a491ae7b admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:04.714378 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3fb43c-2736-40ee-9b7d-b119a491ae7b admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:04.718146 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20f0d49-0e6e-4630-9c80-7dfbf02773c6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:04.720158 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20f0d49-0e6e-4630-9c80-7dfbf02773c6 service nova] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:04.722465 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6a37d6-7d77-49ae-95f4-345354095329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:04.723473 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6a37d6-7d77-49ae-95f4-345354095329 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:04.724043 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6a37d6-7d77-49ae-95f4-345354095329 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:04.730059 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6a37d6-7d77-49ae-95f4-345354095329 admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:04.730059 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6a37d6-7d77-49ae-95f4-345354095329 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:04.730059 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6a37d6-7d77-49ae-95f4-345354095329 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:04.742746 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 863/1734] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:26:04 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:26:04.743477 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 872/1735] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:26:04 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:26:04.747195 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6a37d6-7d77-49ae-95f4-345354095329 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:04.759987 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20f0d49-0e6e-4630-9c80-7dfbf02773c6 service nova] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:04.760437 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6a37d6-7d77-49ae-95f4-345354095329 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:04.762453 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-a20f0d49-0e6e-4630-9c80-7dfbf02773c6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:26:04.773826 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941cd1c3-70b1-4cc5-975e-dbfe6f2b497a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:04.774572 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941cd1c3-70b1-4cc5-975e-dbfe6f2b497a admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:04.775302 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941cd1c3-70b1-4cc5-975e-dbfe6f2b497a admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:04.775302 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941cd1c3-70b1-4cc5-975e-dbfe6f2b497a admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:04.776432 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941cd1c3-70b1-4cc5-975e-dbfe6f2b497a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:04.778566 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941cd1c3-70b1-4cc5-975e-dbfe6f2b497a admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:04.810991 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20f0d49-0e6e-4630-9c80-7dfbf02773c6 service nova] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:04.851950 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f1fbec-676c-4dfd-a24c-b191adeb539d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:04.891114 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f1fbec-676c-4dfd-a24c-b191adeb539d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:04.891518 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f1fbec-676c-4dfd-a24c-b191adeb539d admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:04.893932 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f1fbec-676c-4dfd-a24c-b191adeb539d admin admin] PATH_INFO: `/v3/projects/09d618690e4b4a12b0f0f06ea369f96c` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:04.906205 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 864/1736] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:26:04 2022] GET /identity/v3/roles => generated 1420 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:26:04.907115 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f1fbec-676c-4dfd-a24c-b191adeb539d admin admin] RBAC: Authorizing `identity:delete_project(project_id=09d618690e4b4a12b0f0f06ea369f96c)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:04.909077 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f1fbec-676c-4dfd-a24c-b191adeb539d admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:04.916659 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941cd1c3-70b1-4cc5-975e-dbfe6f2b497a admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:04.933742 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941cd1c3-70b1-4cc5-975e-dbfe6f2b497a admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:04.953982 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ce251b-ade4-43ac-9102-774f9f9d7ae2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:04.954802 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ce251b-ade4-43ac-9102-774f9f9d7ae2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:04.955125 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ce251b-ade4-43ac-9102-774f9f9d7ae2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:04.955596 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ce251b-ade4-43ac-9102-774f9f9d7ae2 admin admin] PATH_INFO: `/v3/projects/f96d1b72ed764675a060748e8e9f5791/users/8bfc35543ede430fbe61009be0eadfbb/roles/60dc7e16aee746d5a3e8aa27d7d8525c` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:04.979836 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ce251b-ade4-43ac-9102-774f9f9d7ae2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f96d1b72ed764675a060748e8e9f5791, user_id=8bfc35543ede430fbe61009be0eadfbb, role_id=60dc7e16aee746d5a3e8aa27d7d8525c)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:04.984220 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ce251b-ade4-43ac-9102-774f9f9d7ae2 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:05.149611 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-a7f1fbec-676c-4dfd-a24c-b191adeb539d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09d618690e4b4a12b0f0f06ea369f96c', 'request_id': 'req-a7f1fbec-676c-4dfd-a24c-b191adeb539d'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:05.162685 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 865/1737] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 24 17:26:04 2022] PUT /identity/v3/projects/f96d1b72ed764675a060748e8e9f5791/users/8bfc35543ede430fbe61009be0eadfbb/roles/60dc7e16aee746d5a3e8aa27d7d8525c => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:26:05.168558 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ce251b-ade4-43ac-9102-774f9f9d7ae2 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:05.272549 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-a7f1fbec-676c-4dfd-a24c-b191adeb539d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09d618690e4b4a12b0f0f06ea369f96c', 'request_id': 'req-a7f1fbec-676c-4dfd-a24c-b191adeb539d'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:05.292208 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-a7f1fbec-676c-4dfd-a24c-b191adeb539d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09d618690e4b4a12b0f0f06ea369f96c', 'request_id': 'req-a7f1fbec-676c-4dfd-a24c-b191adeb539d'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:05.318711 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-a7f1fbec-676c-4dfd-a24c-b191adeb539d admin admin] The token cache is being invalidate because project 09d618690e4b4a12b0f0f06ea369f96c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81731) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:26:05.319363 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-a7f1fbec-676c-4dfd-a24c-b191adeb539d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:05.322551 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-a7f1fbec-676c-4dfd-a24c-b191adeb539d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:05.327208 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 873/1738] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 24 17:26:04 2022] DELETE /identity/v3/projects/09d618690e4b4a12b0f0f06ea369f96c => generated 0 bytes in 568 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:26:05.328340 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f1fbec-676c-4dfd-a24c-b191adeb539d admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:05.331714 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-a7f1fbec-676c-4dfd-a24c-b191adeb539d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:26:05.338949 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ce251b-ade4-43ac-9102-774f9f9d7ae2 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:05.537880 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31c818c-6bf8-4498-bf31-8a2fd2ab4a0b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557aca9255cb4bbb8af52e796c5c5177', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:05.538840 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31c818c-6bf8-4498-bf31-8a2fd2ab4a0b service nova] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:05.538982 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31c818c-6bf8-4498-bf31-8a2fd2ab4a0b service nova] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:05.539250 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31c818c-6bf8-4498-bf31-8a2fd2ab4a0b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:05.548479 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-d31c818c-6bf8-4498-bf31-8a2fd2ab4a0b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:26:05.582518 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f1fbec-676c-4dfd-a24c-b191adeb539d admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:05.601229 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0f7d36-50cf-4a67-91d1-182af99a0eae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:05.601693 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0f7d36-50cf-4a67-91d1-182af99a0eae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:05.601982 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0f7d36-50cf-4a67-91d1-182af99a0eae admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:05.602260 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0f7d36-50cf-4a67-91d1-182af99a0eae admin admin] PATH_INFO: `/v3/projects/a4f0f04f254241e0b0944fdfa80343dc` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:05.607996 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0f7d36-50cf-4a67-91d1-182af99a0eae admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4f0f04f254241e0b0944fdfa80343dc)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:05.609975 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0f7d36-50cf-4a67-91d1-182af99a0eae admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:05.757663 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:26:05.770680 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31c818c-6bf8-4498-bf31-8a2fd2ab4a0b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:05.773099 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31c818c-6bf8-4498-bf31-8a2fd2ab4a0b service nova] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:05.819253 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-9e0f7d36-50cf-4a67-91d1-182af99a0eae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4f0f04f254241e0b0944fdfa80343dc', 'request_id': 'req-9e0f7d36-50cf-4a67-91d1-182af99a0eae'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:05.831017 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 866/1739] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:26:05 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 668 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:26:05.835501 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074a4ade-4e8c-4ac7-9aa6-e25f222cacdf None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:05.835797 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074a4ade-4e8c-4ac7-9aa6-e25f222cacdf None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:05.836080 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074a4ade-4e8c-4ac7-9aa6-e25f222cacdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:05.888715 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-074a4ade-4e8c-4ac7-9aa6-e25f222cacdf None None] MFA Rules not processed for user `ef9e96b709b345c08a4ea10f14211668`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:26:05.888715 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-074a4ade-4e8c-4ac7-9aa6-e25f222cacdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:26:05.951419 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-9e0f7d36-50cf-4a67-91d1-182af99a0eae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4f0f04f254241e0b0944fdfa80343dc', 'request_id': 'req-9e0f7d36-50cf-4a67-91d1-182af99a0eae'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:05.963482 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-9e0f7d36-50cf-4a67-91d1-182af99a0eae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4f0f04f254241e0b0944fdfa80343dc', 'request_id': 'req-9e0f7d36-50cf-4a67-91d1-182af99a0eae'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:05.994886 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-9e0f7d36-50cf-4a67-91d1-182af99a0eae admin admin] The token cache is being invalidate because project a4f0f04f254241e0b0944fdfa80343dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81731) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:26:05.997061 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-9e0f7d36-50cf-4a67-91d1-182af99a0eae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:06.004955 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-9e0f7d36-50cf-4a67-91d1-182af99a0eae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:06.008020 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 874/1740] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 24 17:26:05 2022] DELETE /identity/v3/projects/a4f0f04f254241e0b0944fdfa80343dc => generated 0 bytes in 680 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:26:06.015320 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f23799e-0a10-4c5f-b1df-01c30de72fb2 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:06.015605 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f23799e-0a10-4c5f-b1df-01c30de72fb2 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:06.016167 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f23799e-0a10-4c5f-b1df-01c30de72fb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:06.066158 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 867/1741] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:26:05 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:26:06.071618 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5b58e9-87e8-40a4-99f2-270f0a9de654 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:06.071998 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5b58e9-87e8-40a4-99f2-270f0a9de654 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:06.072290 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5b58e9-87e8-40a4-99f2-270f0a9de654 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:06.132198 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-6f23799e-0a10-4c5f-b1df-01c30de72fb2 None None] MFA Rules not processed for user `8bfc35543ede430fbe61009be0eadfbb`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:26:06.145523 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-6f23799e-0a10-4c5f-b1df-01c30de72fb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:26:06.151858 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-4e5b58e9-87e8-40a4-99f2-270f0a9de654 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:26:06.160394 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-4e5b58e9-87e8-40a4-99f2-270f0a9de654 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:26:06.351626 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 868/1742] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:26:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:26:06.354356 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5b58e9-87e8-40a4-99f2-270f0a9de654 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:06.365870 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-4e5b58e9-87e8-40a4-99f2-270f0a9de654 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:26:06.401978 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 875/1743] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:26:06 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 393 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:26:06.404070 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed624c49-b6f2-4cbf-ba7d-377f94c2ce16 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:06.404367 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed624c49-b6f2-4cbf-ba7d-377f94c2ce16 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:06.404644 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed624c49-b6f2-4cbf-ba7d-377f94c2ce16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:06.415109 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5b58e9-87e8-40a4-99f2-270f0a9de654 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:06.426804 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-ed624c49-b6f2-4cbf-ba7d-377f94c2ce16 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:26:06.428126 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-ed624c49-b6f2-4cbf-ba7d-377f94c2ce16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:26:06.439686 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b4318b-bd22-47f6-b1f5-3c96017839c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:06.441156 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b4318b-bd22-47f6-b1f5-3c96017839c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:06.442806 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b4318b-bd22-47f6-b1f5-3c96017839c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:06.444331 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b4318b-bd22-47f6-b1f5-3c96017839c4 admin admin] PATH_INFO: `/v3/users/71a27715f4ae4d24bbbe4c37a4b6eb3c` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:06.449567 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 876/1744] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:26:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:26:06.450859 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed624c49-b6f2-4cbf-ba7d-377f94c2ce16 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:06.453661 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b4318b-bd22-47f6-b1f5-3c96017839c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=71a27715f4ae4d24bbbe4c37a4b6eb3c)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:06.454349 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-ed624c49-b6f2-4cbf-ba7d-377f94c2ce16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:26:06.456382 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b4318b-bd22-47f6-b1f5-3c96017839c4 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:06.497971 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed624c49-b6f2-4cbf-ba7d-377f94c2ce16 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:06.515082 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15ddf31-b74a-43db-a0e7-24b39afb5745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:06.515981 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15ddf31-b74a-43db-a0e7-24b39afb5745 admin admin] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:06.516429 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15ddf31-b74a-43db-a0e7-24b39afb5745 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:06.516847 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15ddf31-b74a-43db-a0e7-24b39afb5745 admin admin] PATH_INFO: `/v3/projects` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:06.517493 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15ddf31-b74a-43db-a0e7-24b39afb5745 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:06.517961 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:26:06.519636 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15ddf31-b74a-43db-a0e7-24b39afb5745 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:06.676510 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 877/1745] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 24 17:26:06 2022] POST /identity/v3/projects => generated 381 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:26:06.677860 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15ddf31-b74a-43db-a0e7-24b39afb5745 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:06.846546 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-f8b4318b-bd22-47f6-b1f5-3c96017839c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '71a27715f4ae4d24bbbe4c37a4b6eb3c'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:06.898797 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15ddf31-b74a-43db-a0e7-24b39afb5745 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:06.904633 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff14a40-c282-478a-8f4d-6d183b442d07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:06.905260 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff14a40-c282-478a-8f4d-6d183b442d07 admin admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:06.905551 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff14a40-c282-478a-8f4d-6d183b442d07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:06.910642 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff14a40-c282-478a-8f4d-6d183b442d07 admin admin] PATH_INFO: `/v3/domains` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:06.910642 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff14a40-c282-478a-8f4d-6d183b442d07 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:26:06.910642 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff14a40-c282-478a-8f4d-6d183b442d07 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:06.910642 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff14a40-c282-478a-8f4d-6d183b442d07 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:06.963714 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 878/1746] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 24 17:26:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:26:06.968537 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff14a40-c282-478a-8f4d-6d183b442d07 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:07.004540 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff14a40-c282-478a-8f4d-6d183b442d07 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:07.012901 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-f8b4318b-bd22-47f6-b1f5-3c96017839c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '71a27715f4ae4d24bbbe4c37a4b6eb3c'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:07.013083 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acb5234-236d-4ed3-860b-5c8fab102416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:07.013651 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acb5234-236d-4ed3-860b-5c8fab102416 admin admin] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:07.013934 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acb5234-236d-4ed3-860b-5c8fab102416 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:07.016167 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acb5234-236d-4ed3-860b-5c8fab102416 admin admin] PATH_INFO: `/v3/users` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:07.016611 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-f8b4318b-bd22-47f6-b1f5-3c96017839c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '71a27715f4ae4d24bbbe4c37a4b6eb3c'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:07.016701 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acb5234-236d-4ed3-860b-5c8fab102416 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:07.017153 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acb5234-236d-4ed3-860b-5c8fab102416 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:07.027837 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-f8b4318b-bd22-47f6-b1f5-3c96017839c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '71a27715f4ae4d24bbbe4c37a4b6eb3c'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:07.028760 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-f8b4318b-bd22-47f6-b1f5-3c96017839c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '71a27715f4ae4d24bbbe4c37a4b6eb3c'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:07.110229 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 869/1747] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 24 17:26:06 2022] DELETE /identity/v3/users/71a27715f4ae4d24bbbe4c37a4b6eb3c => generated 0 bytes in 758 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:26:07.113763 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b4318b-bd22-47f6-b1f5-3c96017839c4 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:07.116489 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-f8b4318b-bd22-47f6-b1f5-3c96017839c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:26:07.174553 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 879/1748] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 24 17:26:06 2022] POST /identity/v3/users => generated 297 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:26:07.178703 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acb5234-236d-4ed3-860b-5c8fab102416 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:07.205670 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-1acb5234-236d-4ed3-860b-5c8fab102416 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:26:07.254056 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acb5234-236d-4ed3-860b-5c8fab102416 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:07.282058 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b4318b-bd22-47f6-b1f5-3c96017839c4 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:07.290064 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409d260a-46a9-4ae0-9254-cade60c4eadd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:07.291121 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409d260a-46a9-4ae0-9254-cade60c4eadd admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:07.291638 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409d260a-46a9-4ae0-9254-cade60c4eadd admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:07.292094 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409d260a-46a9-4ae0-9254-cade60c4eadd admin admin] PATH_INFO: `/v3/projects` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:07.293697 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409d260a-46a9-4ae0-9254-cade60c4eadd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:07.298287 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409d260a-46a9-4ae0-9254-cade60c4eadd admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:07.358685 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657e7b4e-57b4-4265-a984-a8cdd57aa351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:07.359863 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657e7b4e-57b4-4265-a984-a8cdd57aa351 admin admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:07.360145 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657e7b4e-57b4-4265-a984-a8cdd57aa351 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:07.360409 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657e7b4e-57b4-4265-a984-a8cdd57aa351 admin admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:07.361108 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657e7b4e-57b4-4265-a984-a8cdd57aa351 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:07.363127 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657e7b4e-57b4-4265-a984-a8cdd57aa351 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:07.403554 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 880/1749] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:26:07 2022] GET /identity/v3/roles => generated 1420 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:26:07.404995 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657e7b4e-57b4-4265-a984-a8cdd57aa351 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:07.416325 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657e7b4e-57b4-4265-a984-a8cdd57aa351 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:07.442738 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ce6360-32cd-4809-9d30-94ee075e2425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:07.443582 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ce6360-32cd-4809-9d30-94ee075e2425 admin admin] REQUEST_METHOD: `PUT` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:07.443862 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ce6360-32cd-4809-9d30-94ee075e2425 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:07.444234 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ce6360-32cd-4809-9d30-94ee075e2425 admin admin] PATH_INFO: `/v3/projects/6f663c2438eb48a5a861ae6d9524be47/users/d8be0791e2b740dfba62612d5a64247f/roles/c5346bc54abf445f9900edf94264c850` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:07.476424 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 870/1750] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 24 17:26:07 2022] POST /identity/v3/projects => generated 385 bytes in 365 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:26:07.481647 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409d260a-46a9-4ae0-9254-cade60c4eadd admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:07.486112 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ce6360-32cd-4809-9d30-94ee075e2425 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f663c2438eb48a5a861ae6d9524be47, user_id=d8be0791e2b740dfba62612d5a64247f, role_id=c5346bc54abf445f9900edf94264c850)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:07.491003 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ce6360-32cd-4809-9d30-94ee075e2425 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:07.624852 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 881/1751] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 24 17:26:07 2022] PUT /identity/v3/projects/6f663c2438eb48a5a861ae6d9524be47/users/d8be0791e2b740dfba62612d5a64247f/roles/c5346bc54abf445f9900edf94264c850 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:26:07.626961 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ce6360-32cd-4809-9d30-94ee075e2425 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:07.721257 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409d260a-46a9-4ae0-9254-cade60c4eadd admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:07.829331 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ce6360-32cd-4809-9d30-94ee075e2425 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:07.847625 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99940d9-8cfe-4020-8171-d0dc09b7b48f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:07.849919 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99940d9-8cfe-4020-8171-d0dc09b7b48f admin admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:07.849919 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99940d9-8cfe-4020-8171-d0dc09b7b48f admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:07.849919 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99940d9-8cfe-4020-8171-d0dc09b7b48f admin admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:07.849919 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99940d9-8cfe-4020-8171-d0dc09b7b48f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:07.853011 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99940d9-8cfe-4020-8171-d0dc09b7b48f admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:07.880433 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 882/1752] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:26:07 2022] GET /identity/v3/roles => generated 1420 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:26:07.884832 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99940d9-8cfe-4020-8171-d0dc09b7b48f admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:07.885088 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e8e923-b333-4f45-9451-68816845b847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:07.885742 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e8e923-b333-4f45-9451-68816845b847 admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:07.885935 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e8e923-b333-4f45-9451-68816845b847 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:07.886213 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e8e923-b333-4f45-9451-68816845b847 admin admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:07.891438 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e8e923-b333-4f45-9451-68816845b847 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:07.910700 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99940d9-8cfe-4020-8171-d0dc09b7b48f admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:07.920429 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e8e923-b333-4f45-9451-68816845b847 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:07.923773 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962333dd-17a4-4b36-aae5-fcd45c6730b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:07.924647 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962333dd-17a4-4b36-aae5-fcd45c6730b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:07.924933 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962333dd-17a4-4b36-aae5-fcd45c6730b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:07.925209 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962333dd-17a4-4b36-aae5-fcd45c6730b2 admin admin] PATH_INFO: `/v3/projects/6f663c2438eb48a5a861ae6d9524be47/users/d8be0791e2b740dfba62612d5a64247f/roles/c5346bc54abf445f9900edf94264c850` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:07.929411 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962333dd-17a4-4b36-aae5-fcd45c6730b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f663c2438eb48a5a861ae6d9524be47, user_id=d8be0791e2b740dfba62612d5a64247f, role_id=c5346bc54abf445f9900edf94264c850)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:07.935099 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962333dd-17a4-4b36-aae5-fcd45c6730b2 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:07.971370 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 883/1753] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 24 17:26:07 2022] PUT /identity/v3/projects/6f663c2438eb48a5a861ae6d9524be47/users/d8be0791e2b740dfba62612d5a64247f/roles/c5346bc54abf445f9900edf94264c850 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:26:07.974111 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9d423b-f9bd-49e2-9b2c-55ded33455c6 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:07.974410 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9d423b-f9bd-49e2-9b2c-55ded33455c6 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:07.974662 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9d423b-f9bd-49e2-9b2c-55ded33455c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:08.026674 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:26:08.077828 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-5c9d423b-f9bd-49e2-9b2c-55ded33455c6 None None] MFA Rules not processed for user `d8be0791e2b740dfba62612d5a64247f`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:26:08.079160 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-5c9d423b-f9bd-49e2-9b2c-55ded33455c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:26:08.101070 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 871/1754] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 24 17:26:07 2022] POST /identity/v3/users => generated 293 bytes in 620 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:26:08.106141 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e8e923-b333-4f45-9451-68816845b847 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:08.139172 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e8e923-b333-4f45-9451-68816845b847 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:08.176140 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:26:08.184894 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 884/1755] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:26:07 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:26:08.202355 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9d423b-f9bd-49e2-9b2c-55ded33455c6 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:08.205074 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-5c9d423b-f9bd-49e2-9b2c-55ded33455c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:26:08.284975 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcfdc92-f272-4b9f-80f5-1edb1fe583b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:08.289103 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcfdc92-f272-4b9f-80f5-1edb1fe583b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:08.289609 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcfdc92-f272-4b9f-80f5-1edb1fe583b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:08.290067 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcfdc92-f272-4b9f-80f5-1edb1fe583b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:08.291211 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcfdc92-f272-4b9f-80f5-1edb1fe583b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:08.294653 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcfdc92-f272-4b9f-80f5-1edb1fe583b4 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:08.310877 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9d423b-f9bd-49e2-9b2c-55ded33455c6 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:08.315387 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 872/1756] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:26:08 2022] GET /identity/v3/roles => generated 1420 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:26:08.317377 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcfdc92-f272-4b9f-80f5-1edb1fe583b4 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:08.320645 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa609da3-5d41-4f15-b9db-e577104a5e80 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be06246d47584b529150ceb98a5cfd95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:08.321281 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa609da3-5d41-4f15-b9db-e577104a5e80 service cinder] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:08.326593 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa609da3-5d41-4f15-b9db-e577104a5e80 service cinder] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:08.327184 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa609da3-5d41-4f15-b9db-e577104a5e80 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:08.342002 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcfdc92-f272-4b9f-80f5-1edb1fe583b4 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:08.352201 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa609da3-5d41-4f15-b9db-e577104a5e80 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:08.355110 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa609da3-5d41-4f15-b9db-e577104a5e80 service cinder] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:08.358018 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41d2d3c-00bc-458a-8c47-f841fdaddb8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:08.358741 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41d2d3c-00bc-458a-8c47-f841fdaddb8a admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:08.359980 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41d2d3c-00bc-458a-8c47-f841fdaddb8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:08.360304 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41d2d3c-00bc-458a-8c47-f841fdaddb8a admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:08.363446 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41d2d3c-00bc-458a-8c47-f841fdaddb8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:08.369729 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41d2d3c-00bc-458a-8c47-f841fdaddb8a admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:08.407530 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 885/1757] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:26:08 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:26:08.415751 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 873/1758] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:26:08 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:26:08.417634 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa609da3-5d41-4f15-b9db-e577104a5e80 service cinder] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:08.437371 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa609da3-5d41-4f15-b9db-e577104a5e80 service cinder] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:08.451159 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87281e95-8ebe-422a-88e9-5d6160fe3250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:08.453640 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87281e95-8ebe-422a-88e9-5d6160fe3250 admin admin] REQUEST_METHOD: `PUT` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:08.454172 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87281e95-8ebe-422a-88e9-5d6160fe3250 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:08.454450 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87281e95-8ebe-422a-88e9-5d6160fe3250 admin admin] PATH_INFO: `/v3/projects/aba808fdeca64e9685a823ba97626f26/users/059b6c74b3d74c658b637473bdd14d50/roles/60dc7e16aee746d5a3e8aa27d7d8525c` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:08.494422 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87281e95-8ebe-422a-88e9-5d6160fe3250 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aba808fdeca64e9685a823ba97626f26, user_id=059b6c74b3d74c658b637473bdd14d50, role_id=60dc7e16aee746d5a3e8aa27d7d8525c)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:08.498803 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87281e95-8ebe-422a-88e9-5d6160fe3250 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:08.586227 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 886/1759] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 24 17:26:08 2022] PUT /identity/v3/projects/aba808fdeca64e9685a823ba97626f26/users/059b6c74b3d74c658b637473bdd14d50/roles/60dc7e16aee746d5a3e8aa27d7d8525c => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:26:08.600349 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c395011-4e55-47df-b7c6-19cbf12e0ac0 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:08.600642 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c395011-4e55-47df-b7c6-19cbf12e0ac0 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:08.600911 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c395011-4e55-47df-b7c6-19cbf12e0ac0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:08.706188 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-0c395011-4e55-47df-b7c6-19cbf12e0ac0 None None] MFA Rules not processed for user `059b6c74b3d74c658b637473bdd14d50`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:26:08.708274 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-0c395011-4e55-47df-b7c6-19cbf12e0ac0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:26:08.752084 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0b3e92-fea3-4aaa-9966-36d03bbea2bc None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:08.752582 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0b3e92-fea3-4aaa-9966-36d03bbea2bc None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:08.752861 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0b3e92-fea3-4aaa-9966-36d03bbea2bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:08.989388 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-2d0b3e92-fea3-4aaa-9966-36d03bbea2bc None None] MFA Rules not processed for user `0ccc6723f7514e52863d102de3839951`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:26:08.996647 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-2d0b3e92-fea3-4aaa-9966-36d03bbea2bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:26:09.013995 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 874/1760] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:26:08 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 415 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:26:09.020404 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c395011-4e55-47df-b7c6-19cbf12e0ac0 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:09.249634 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:26:09.280059 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c395011-4e55-47df-b7c6-19cbf12e0ac0 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:09.298284 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193e51bb-7a63-41a0-99c8-abef87775b06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:09.301101 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:26:09.302169 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 887/1761] 10.222.0.18 () {58 vars in 1009 bytes} [Wed Aug 24 17:26:08 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 551 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:26:09.318730 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193e51bb-7a63-41a0-99c8-abef87775b06 admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:09.319554 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193e51bb-7a63-41a0-99c8-abef87775b06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:09.320062 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193e51bb-7a63-41a0-99c8-abef87775b06 admin admin] PATH_INFO: `/v3/projects` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:09.320744 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193e51bb-7a63-41a0-99c8-abef87775b06 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:09.322726 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193e51bb-7a63-41a0-99c8-abef87775b06 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:09.502141 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 875/1762] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 24 17:26:09 2022] POST /identity/v3/projects => generated 387 bytes in 481 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:26:09.504681 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0b3e92-fea3-4aaa-9966-36d03bbea2bc None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:09.520106 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0b3e92-fea3-4aaa-9966-36d03bbea2bc None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:09.719798 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70eae06f-8424-469b-9f76-2ee562e6ec03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:09.720889 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eae06f-8424-469b-9f76-2ee562e6ec03 admin admin] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:09.721628 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eae06f-8424-469b-9f76-2ee562e6ec03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:09.722376 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eae06f-8424-469b-9f76-2ee562e6ec03 admin admin] PATH_INFO: `/v3/users` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:09.723263 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70eae06f-8424-469b-9f76-2ee562e6ec03 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:09.725556 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70eae06f-8424-469b-9f76-2ee562e6ec03 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:09.844039 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193e51bb-7a63-41a0-99c8-abef87775b06 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:09.844717 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-193e51bb-7a63-41a0-99c8-abef87775b06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:26:09.859074 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 888/1763] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 24 17:26:09 2022] POST /identity/v3/users => generated 300 bytes in 355 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:26:09.862742 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70eae06f-8424-469b-9f76-2ee562e6ec03 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:09.888475 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70eae06f-8424-469b-9f76-2ee562e6ec03 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:09.910054 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca2d007-5041-4227-ba0d-93948ba8e4c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:09.911182 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca2d007-5041-4227-ba0d-93948ba8e4c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:09.916516 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca2d007-5041-4227-ba0d-93948ba8e4c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:09.916516 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca2d007-5041-4227-ba0d-93948ba8e4c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:09.916516 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca2d007-5041-4227-ba0d-93948ba8e4c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:09.925921 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca2d007-5041-4227-ba0d-93948ba8e4c7 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:09.961453 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 889/1764] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:26:09 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:26:09.972087 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca2d007-5041-4227-ba0d-93948ba8e4c7 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:09.983457 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca2d007-5041-4227-ba0d-93948ba8e4c7 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:09.994362 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9637916e-b626-4dd6-acef-33592f3784c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:09.994866 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9637916e-b626-4dd6-acef-33592f3784c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:09.995490 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9637916e-b626-4dd6-acef-33592f3784c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:09.996324 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9637916e-b626-4dd6-acef-33592f3784c6 admin admin] PATH_INFO: `/v3/projects/7d7968fbdd3c44af8c4b12d9b0cfa5a9/users/780a0cd6a12a493ca52183d2740ca488/roles/c5346bc54abf445f9900edf94264c850` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:10.031195 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193e51bb-7a63-41a0-99c8-abef87775b06 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:10.031850 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9637916e-b626-4dd6-acef-33592f3784c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d7968fbdd3c44af8c4b12d9b0cfa5a9, user_id=780a0cd6a12a493ca52183d2740ca488, role_id=c5346bc54abf445f9900edf94264c850)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:10.039730 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9637916e-b626-4dd6-acef-33592f3784c6 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:10.043376 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ba9e1a-b842-4bc3-ade7-79a152200b76 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef8caa353be34499a58cd25b72ee4032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:10.044359 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ba9e1a-b842-4bc3-ade7-79a152200b76 service swift] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:10.045296 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ba9e1a-b842-4bc3-ade7-79a152200b76 service swift] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:10.045714 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ba9e1a-b842-4bc3-ade7-79a152200b76 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:10.174216 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 890/1765] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 24 17:26:09 2022] PUT /identity/v3/projects/7d7968fbdd3c44af8c4b12d9b0cfa5a9/users/780a0cd6a12a493ca52183d2740ca488/roles/c5346bc54abf445f9900edf94264c850 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:26:10.175647 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9637916e-b626-4dd6-acef-33592f3784c6 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:10.187754 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ba9e1a-b842-4bc3-ade7-79a152200b76 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:10.191819 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ba9e1a-b842-4bc3-ade7-79a152200b76 service swift] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:10.218930 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 876/1766] 10.222.0.18 () {60 vars in 1384 bytes} [Wed Aug 24 17:26:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 24 17:26:10.369086 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ba9e1a-b842-4bc3-ade7-79a152200b76 service swift] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:10.551695 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9637916e-b626-4dd6-acef-33592f3784c6 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:10.570508 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f1d658-38cf-4b6e-9278-4076a24381a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:10.570508 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f1d658-38cf-4b6e-9278-4076a24381a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:10.570508 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f1d658-38cf-4b6e-9278-4076a24381a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:10.570508 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f1d658-38cf-4b6e-9278-4076a24381a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:10.571060 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f1d658-38cf-4b6e-9278-4076a24381a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:10.575236 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f1d658-38cf-4b6e-9278-4076a24381a6 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:10.587165 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 891/1767] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:26:10 2022] GET /identity/v3/roles => generated 1420 bytes in 412 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:26:10.598940 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f1d658-38cf-4b6e-9278-4076a24381a6 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:10.619679 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f1d658-38cf-4b6e-9278-4076a24381a6 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:10.626493 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ba9e1a-b842-4bc3-ade7-79a152200b76 service swift] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:10.645814 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab843c41-89fb-48c4-9c95-09f6d0195c5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:10.647166 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab843c41-89fb-48c4-9c95-09f6d0195c5f admin admin] REQUEST_METHOD: `PUT` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:10.648993 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab843c41-89fb-48c4-9c95-09f6d0195c5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:10.649639 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab843c41-89fb-48c4-9c95-09f6d0195c5f admin admin] PATH_INFO: `/v3/projects/7d7968fbdd3c44af8c4b12d9b0cfa5a9/users/780a0cd6a12a493ca52183d2740ca488/roles/c5346bc54abf445f9900edf94264c850` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:10.656835 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab843c41-89fb-48c4-9c95-09f6d0195c5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d7968fbdd3c44af8c4b12d9b0cfa5a9, user_id=780a0cd6a12a493ca52183d2740ca488, role_id=c5346bc54abf445f9900edf94264c850)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:10.664901 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab843c41-89fb-48c4-9c95-09f6d0195c5f admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:10.698675 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 892/1768] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 24 17:26:10 2022] PUT /identity/v3/projects/7d7968fbdd3c44af8c4b12d9b0cfa5a9/users/780a0cd6a12a493ca52183d2740ca488/roles/c5346bc54abf445f9900edf94264c850 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:26:10.701659 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab843c41-89fb-48c4-9c95-09f6d0195c5f admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:10.710062 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-ab843c41-89fb-48c4-9c95-09f6d0195c5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:26:10.869186 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a752fa9-abae-43b0-836d-b57516340803 tempest-VolumeQuotaClassesTest-1918764259 tempest-VolumeQuotaClassesTest-1918764259-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8be0791e2b740dfba62612d5a64247f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f663c2438eb48a5a861ae6d9524be47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:10.878076 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:DELETE=8,keystone:SELECT=90,keystone:INSERT=2 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:26:10.879264 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a752fa9-abae-43b0-836d-b57516340803 tempest-VolumeQuotaClassesTest-1918764259 tempest-VolumeQuotaClassesTest-1918764259-project-admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:10.881934 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a752fa9-abae-43b0-836d-b57516340803 tempest-VolumeQuotaClassesTest-1918764259 tempest-VolumeQuotaClassesTest-1918764259-project-admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:10.882339 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a752fa9-abae-43b0-836d-b57516340803 tempest-VolumeQuotaClassesTest-1918764259 tempest-VolumeQuotaClassesTest-1918764259-project-admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:10.888752 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a752fa9-abae-43b0-836d-b57516340803 tempest-VolumeQuotaClassesTest-1918764259 tempest-VolumeQuotaClassesTest-1918764259-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:26:10.889195 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a752fa9-abae-43b0-836d-b57516340803 tempest-VolumeQuotaClassesTest-1918764259 tempest-VolumeQuotaClassesTest-1918764259-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:10.912238 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a752fa9-abae-43b0-836d-b57516340803 tempest-VolumeQuotaClassesTest-1918764259 tempest-VolumeQuotaClassesTest-1918764259-project-admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:10.929861 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab843c41-89fb-48c4-9c95-09f6d0195c5f admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:10.943874 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcbf9a6-908b-4d91-bb5b-bd316fff89ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:10.949491 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcbf9a6-908b-4d91-bb5b-bd316fff89ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:10.949785 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcbf9a6-908b-4d91-bb5b-bd316fff89ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:10.950036 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcbf9a6-908b-4d91-bb5b-bd316fff89ea admin admin] PATH_INFO: `/v3/projects/4209c16dbbda405ca4cdca13c1427caa` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:10.950679 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 877/1769] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 24 17:26:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 582 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:26:10.956629 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669e0d4e-850f-4d4d-8299-0f0fcdd38d68 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:10.957242 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669e0d4e-850f-4d4d-8299-0f0fcdd38d68 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:10.957741 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669e0d4e-850f-4d4d-8299-0f0fcdd38d68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:10.959300 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcbf9a6-908b-4d91-bb5b-bd316fff89ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=4209c16dbbda405ca4cdca13c1427caa)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:10.965932 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcbf9a6-908b-4d91-bb5b-bd316fff89ea admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:11.008795 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-669e0d4e-850f-4d4d-8299-0f0fcdd38d68 None None] MFA Rules not processed for user `780a0cd6a12a493ca52183d2740ca488`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:26:11.011007 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-669e0d4e-850f-4d4d-8299-0f0fcdd38d68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:26:11.283156 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 878/1770] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:26:10 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 323 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:26:11.283156 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669e0d4e-850f-4d4d-8299-0f0fcdd38d68 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:11.295676 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-2bcbf9a6-908b-4d91-bb5b-bd316fff89ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4209c16dbbda405ca4cdca13c1427caa', 'request_id': 'req-2bcbf9a6-908b-4d91-bb5b-bd316fff89ea'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:11.392592 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-2bcbf9a6-908b-4d91-bb5b-bd316fff89ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4209c16dbbda405ca4cdca13c1427caa', 'request_id': 'req-2bcbf9a6-908b-4d91-bb5b-bd316fff89ea'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:11.401134 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-2bcbf9a6-908b-4d91-bb5b-bd316fff89ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4209c16dbbda405ca4cdca13c1427caa', 'request_id': 'req-2bcbf9a6-908b-4d91-bb5b-bd316fff89ea'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:11.414953 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669e0d4e-850f-4d4d-8299-0f0fcdd38d68 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:11.417551 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-2bcbf9a6-908b-4d91-bb5b-bd316fff89ea admin admin] The token cache is being invalidate because project 4209c16dbbda405ca4cdca13c1427caa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81731) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:26:11.418184 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-2bcbf9a6-908b-4d91-bb5b-bd316fff89ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:11.419299 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-2bcbf9a6-908b-4d91-bb5b-bd316fff89ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:11.422134 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 893/1771] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 24 17:26:10 2022] DELETE /identity/v3/projects/4209c16dbbda405ca4cdca13c1427caa => generated 0 bytes in 722 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:26:11.423989 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcbf9a6-908b-4d91-bb5b-bd316fff89ea admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:11.425780 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-2bcbf9a6-908b-4d91-bb5b-bd316fff89ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:26:11.534250 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcbf9a6-908b-4d91-bb5b-bd316fff89ea admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:11.538945 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab6b58f-e845-4c30-94f4-e62b69f0ba57 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be06246d47584b529150ceb98a5cfd95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:11.541055 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab6b58f-e845-4c30-94f4-e62b69f0ba57 service cinder] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:11.541055 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab6b58f-e845-4c30-94f4-e62b69f0ba57 service cinder] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:11.541055 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab6b58f-e845-4c30-94f4-e62b69f0ba57 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:11.549366 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7b69db-5bef-42d8-8eea-2e9df7d449cf tempest-VolumeQuotaClassesTest-1918764259 tempest-VolumeQuotaClassesTest-1918764259-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8be0791e2b740dfba62612d5a64247f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f663c2438eb48a5a861ae6d9524be47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:11.550435 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-4ab6b58f-e845-4c30-94f4-e62b69f0ba57 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:26:11.550750 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7b69db-5bef-42d8-8eea-2e9df7d449cf tempest-VolumeQuotaClassesTest-1918764259 tempest-VolumeQuotaClassesTest-1918764259-project-admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:11.551232 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7b69db-5bef-42d8-8eea-2e9df7d449cf tempest-VolumeQuotaClassesTest-1918764259 tempest-VolumeQuotaClassesTest-1918764259-project-admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:11.551665 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7b69db-5bef-42d8-8eea-2e9df7d449cf tempest-VolumeQuotaClassesTest-1918764259 tempest-VolumeQuotaClassesTest-1918764259-project-admin] PATH_INFO: `/v3/projects` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:11.552858 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7b69db-5bef-42d8-8eea-2e9df7d449cf tempest-VolumeQuotaClassesTest-1918764259 tempest-VolumeQuotaClassesTest-1918764259-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:11.555764 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7b69db-5bef-42d8-8eea-2e9df7d449cf tempest-VolumeQuotaClassesTest-1918764259 tempest-VolumeQuotaClassesTest-1918764259-project-admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:11.738266 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab6b58f-e845-4c30-94f4-e62b69f0ba57 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:11.742683 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab6b58f-e845-4c30-94f4-e62b69f0ba57 service cinder] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:11.761029 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 879/1772] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 24 17:26:11 2022] POST /identity/v3/projects => generated 355 bytes in 484 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:26:11.769226 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df167049-60fb-4923-8e36-62090f87f60a None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:11.769969 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df167049-60fb-4923-8e36-62090f87f60a None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:11.771289 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df167049-60fb-4923-8e36-62090f87f60a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:11.782812 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 894/1773] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:26:11 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:26:11.784045 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab6b58f-e845-4c30-94f4-e62b69f0ba57 service cinder] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:11.805156 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-4ab6b58f-e845-4c30-94f4-e62b69f0ba57 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:26:11.811271 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-df167049-60fb-4923-8e36-62090f87f60a None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:26:11.814015 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-df167049-60fb-4923-8e36-62090f87f60a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:26:11.833418 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:26:11.854578 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab6b58f-e845-4c30-94f4-e62b69f0ba57 service cinder] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:12.023486 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 880/1774] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:26:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:26:12.024362 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df167049-60fb-4923-8e36-62090f87f60a None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:12.033901 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-df167049-60fb-4923-8e36-62090f87f60a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:26:12.061201 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df167049-60fb-4923-8e36-62090f87f60a None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:12.069495 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e854b3-0e6a-46b2-aa2e-ff47e95ca6f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:12.144210 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e854b3-0e6a-46b2-aa2e-ff47e95ca6f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:12.144210 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e854b3-0e6a-46b2-aa2e-ff47e95ca6f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:12.144210 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e854b3-0e6a-46b2-aa2e-ff47e95ca6f3 admin admin] PATH_INFO: `/v3/users/059b6c74b3d74c658b637473bdd14d50` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:12.144210 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e854b3-0e6a-46b2-aa2e-ff47e95ca6f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=059b6c74b3d74c658b637473bdd14d50)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:12.144210 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e854b3-0e6a-46b2-aa2e-ff47e95ca6f3 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:12.145591 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc5b14c-f448-4978-a508-9c610e6ad9ea tempest-VolumeQuotaClassesTest-1918764259 tempest-VolumeQuotaClassesTest-1918764259-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8be0791e2b740dfba62612d5a64247f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f663c2438eb48a5a861ae6d9524be47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:12.145591 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc5b14c-f448-4978-a508-9c610e6ad9ea tempest-VolumeQuotaClassesTest-1918764259 tempest-VolumeQuotaClassesTest-1918764259-project-admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:12.145591 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc5b14c-f448-4978-a508-9c610e6ad9ea tempest-VolumeQuotaClassesTest-1918764259 tempest-VolumeQuotaClassesTest-1918764259-project-admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:12.146274 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc5b14c-f448-4978-a508-9c610e6ad9ea tempest-VolumeQuotaClassesTest-1918764259 tempest-VolumeQuotaClassesTest-1918764259-project-admin] PATH_INFO: `/v3/domains` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:12.146274 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc5b14c-f448-4978-a508-9c610e6ad9ea tempest-VolumeQuotaClassesTest-1918764259 tempest-VolumeQuotaClassesTest-1918764259-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:26:12.146274 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc5b14c-f448-4978-a508-9c610e6ad9ea tempest-VolumeQuotaClassesTest-1918764259 tempest-VolumeQuotaClassesTest-1918764259-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:12.146274 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc5b14c-f448-4978-a508-9c610e6ad9ea tempest-VolumeQuotaClassesTest-1918764259 tempest-VolumeQuotaClassesTest-1918764259-project-admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:12.158458 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 895/1775] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 24 17:26:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 375 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:26:12.160128 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0ebc00-aacd-4580-a67a-6ae94ab23c6b None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:12.160831 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0ebc00-aacd-4580-a67a-6ae94ab23c6b None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:12.160831 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0ebc00-aacd-4580-a67a-6ae94ab23c6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:12.192739 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-cb0ebc00-aacd-4580-a67a-6ae94ab23c6b None None] MFA Rules not processed for user `0e9ed4d47aef4140bfb2d3bb798cc559`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:26:12.195557 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-cb0ebc00-aacd-4580-a67a-6ae94ab23c6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:26:12.289700 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 896/1776] 10.222.0.50 () {58 vars in 1015 bytes} [Wed Aug 24 17:26:12 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:26:12.293398 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484654bd-b571-463c-be49-58e8fa80531e None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:12.294005 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484654bd-b571-463c-be49-58e8fa80531e None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:12.294505 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484654bd-b571-463c-be49-58e8fa80531e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:12.326357 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-484654bd-b571-463c-be49-58e8fa80531e None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:26:12.328139 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-484654bd-b571-463c-be49-58e8fa80531e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:26:12.342986 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 897/1777] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:26:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:26:12.342986 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484654bd-b571-463c-be49-58e8fa80531e None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:12.359170 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484654bd-b571-463c-be49-58e8fa80531e None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:12.365903 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8521c7-0e42-4960-93f4-84033e9524d0 tempest-VolumeQuotaClassesTest-1918764259 tempest-VolumeQuotaClassesTest-1918764259-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd8be0791e2b740dfba62612d5a64247f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f663c2438eb48a5a861ae6d9524be47', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:12.367170 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8521c7-0e42-4960-93f4-84033e9524d0 tempest-VolumeQuotaClassesTest-1918764259 tempest-VolumeQuotaClassesTest-1918764259-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:12.367634 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8521c7-0e42-4960-93f4-84033e9524d0 tempest-VolumeQuotaClassesTest-1918764259 tempest-VolumeQuotaClassesTest-1918764259-project-admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:12.368228 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8521c7-0e42-4960-93f4-84033e9524d0 tempest-VolumeQuotaClassesTest-1918764259 tempest-VolumeQuotaClassesTest-1918764259-project-admin] PATH_INFO: `/v3/projects/7315380ad2344c8d83c1a8880538ab81` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:12.369805 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8521c7-0e42-4960-93f4-84033e9524d0 tempest-VolumeQuotaClassesTest-1918764259 tempest-VolumeQuotaClassesTest-1918764259-project-admin] RBAC: Authorizing `identity:delete_project(project_id=7315380ad2344c8d83c1a8880538ab81)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:12.372465 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8521c7-0e42-4960-93f4-84033e9524d0 tempest-VolumeQuotaClassesTest-1918764259 tempest-VolumeQuotaClassesTest-1918764259-project-admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:12.498941 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-f3e854b3-0e6a-46b2-aa2e-ff47e95ca6f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '059b6c74b3d74c658b637473bdd14d50'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:12.520623 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-6d8521c7-0e42-4960-93f4-84033e9524d0 tempest-VolumeQuotaClassesTest-1918764259 tempest-VolumeQuotaClassesTest-1918764259-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7315380ad2344c8d83c1a8880538ab81', 'request_id': 'req-6d8521c7-0e42-4960-93f4-84033e9524d0'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:12.614351 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-f3e854b3-0e6a-46b2-aa2e-ff47e95ca6f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '059b6c74b3d74c658b637473bdd14d50'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:12.619264 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-f3e854b3-0e6a-46b2-aa2e-ff47e95ca6f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '059b6c74b3d74c658b637473bdd14d50'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:12.646258 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-f3e854b3-0e6a-46b2-aa2e-ff47e95ca6f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '059b6c74b3d74c658b637473bdd14d50'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:12.648774 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-f3e854b3-0e6a-46b2-aa2e-ff47e95ca6f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '059b6c74b3d74c658b637473bdd14d50'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:12.700919 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 881/1778] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 24 17:26:12 2022] DELETE /identity/v3/users/059b6c74b3d74c658b637473bdd14d50 => generated 0 bytes in 673 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:26:12.701126 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-6d8521c7-0e42-4960-93f4-84033e9524d0 tempest-VolumeQuotaClassesTest-1918764259 tempest-VolumeQuotaClassesTest-1918764259-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7315380ad2344c8d83c1a8880538ab81', 'request_id': 'req-6d8521c7-0e42-4960-93f4-84033e9524d0'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:12.704013 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e854b3-0e6a-46b2-aa2e-ff47e95ca6f3 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:12.707816 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-f3e854b3-0e6a-46b2-aa2e-ff47e95ca6f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:26:12.710297 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-6d8521c7-0e42-4960-93f4-84033e9524d0 tempest-VolumeQuotaClassesTest-1918764259 tempest-VolumeQuotaClassesTest-1918764259-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7315380ad2344c8d83c1a8880538ab81', 'request_id': 'req-6d8521c7-0e42-4960-93f4-84033e9524d0'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:12.733755 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-6d8521c7-0e42-4960-93f4-84033e9524d0 tempest-VolumeQuotaClassesTest-1918764259 tempest-VolumeQuotaClassesTest-1918764259-project-admin] The token cache is being invalidate because project 7315380ad2344c8d83c1a8880538ab81 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81731) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:26:12.735621 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-6d8521c7-0e42-4960-93f4-84033e9524d0 tempest-VolumeQuotaClassesTest-1918764259 tempest-VolumeQuotaClassesTest-1918764259-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:12.736774 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-6d8521c7-0e42-4960-93f4-84033e9524d0 tempest-VolumeQuotaClassesTest-1918764259 tempest-VolumeQuotaClassesTest-1918764259-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:12.743504 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 898/1779] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 24 17:26:12 2022] DELETE /identity/v3/projects/7315380ad2344c8d83c1a8880538ab81 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:26:12.745022 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8521c7-0e42-4960-93f4-84033e9524d0 tempest-VolumeQuotaClassesTest-1918764259 tempest-VolumeQuotaClassesTest-1918764259-project-admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:12.747431 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-6d8521c7-0e42-4960-93f4-84033e9524d0 tempest-VolumeQuotaClassesTest-1918764259 tempest-VolumeQuotaClassesTest-1918764259-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:26:12.881472 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e854b3-0e6a-46b2-aa2e-ff47e95ca6f3 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:12.887677 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5a0aae-2ebb-497f-9ac2-6e8213d870ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e9ed4d47aef4140bfb2d3bb798cc559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:12.888476 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5a0aae-2ebb-497f-9ac2-6e8213d870ad service neutron] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:12.888642 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5a0aae-2ebb-497f-9ac2-6e8213d870ad service neutron] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:12.889161 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5a0aae-2ebb-497f-9ac2-6e8213d870ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:12.892110 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-fe5a0aae-2ebb-497f-9ac2-6e8213d870ad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:26:12.914030 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5a0aae-2ebb-497f-9ac2-6e8213d870ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:12.914030 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5a0aae-2ebb-497f-9ac2-6e8213d870ad service neutron] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:12.917496 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8521c7-0e42-4960-93f4-84033e9524d0 tempest-VolumeQuotaClassesTest-1918764259 tempest-VolumeQuotaClassesTest-1918764259-project-admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:12.920591 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 882/1780] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:26:12 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:26:12.922768 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25169ffb-966c-45b9-8496-10bf1bafee25 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:12.923317 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25169ffb-966c-45b9-8496-10bf1bafee25 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:12.923867 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25169ffb-966c-45b9-8496-10bf1bafee25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:12.982202 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-25169ffb-966c-45b9-8496-10bf1bafee25 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:26:12.983676 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-25169ffb-966c-45b9-8496-10bf1bafee25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:26:13.058214 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:26:13.100199 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d47ac1c-528e-4f32-a768-7e24a5019f48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:13.100944 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d47ac1c-528e-4f32-a768-7e24a5019f48 admin admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:13.101224 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d47ac1c-528e-4f32-a768-7e24a5019f48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:13.101496 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d47ac1c-528e-4f32-a768-7e24a5019f48 admin admin] PATH_INFO: `/v3/domains` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:13.102171 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d47ac1c-528e-4f32-a768-7e24a5019f48 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:26:13.102521 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d47ac1c-528e-4f32-a768-7e24a5019f48 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:13.104441 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d47ac1c-528e-4f32-a768-7e24a5019f48 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:13.133619 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 899/1781] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 24 17:26:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 389 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:26:13.134686 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d47ac1c-528e-4f32-a768-7e24a5019f48 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:13.159771 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d47ac1c-528e-4f32-a768-7e24a5019f48 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:13.169229 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66e6f82-656f-4b82-ae86-8b9e7ae4a3ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:13.172780 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66e6f82-656f-4b82-ae86-8b9e7ae4a3ad admin admin] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:13.172780 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66e6f82-656f-4b82-ae86-8b9e7ae4a3ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:13.172780 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66e6f82-656f-4b82-ae86-8b9e7ae4a3ad admin admin] PATH_INFO: `/v3/projects` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:13.172780 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66e6f82-656f-4b82-ae86-8b9e7ae4a3ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:13.175879 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66e6f82-656f-4b82-ae86-8b9e7ae4a3ad admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:13.177705 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 883/1782] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:26:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:26:13.348938 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25169ffb-966c-45b9-8496-10bf1bafee25 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:13.350688 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-25169ffb-966c-45b9-8496-10bf1bafee25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:26:13.388448 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25169ffb-966c-45b9-8496-10bf1bafee25 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:13.404174 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:26:13.414142 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 900/1783] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 24 17:26:13 2022] POST /identity/v3/projects => generated 383 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:26:13.415101 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28444a4d-c0a1-4868-a3ae-f4b0da2f92fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e9ed4d47aef4140bfb2d3bb798cc559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:13.416216 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28444a4d-c0a1-4868-a3ae-f4b0da2f92fb service neutron] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:13.421130 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66e6f82-656f-4b82-ae86-8b9e7ae4a3ad admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:13.425534 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28444a4d-c0a1-4868-a3ae-f4b0da2f92fb service neutron] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:13.425925 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28444a4d-c0a1-4868-a3ae-f4b0da2f92fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:13.475692 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28444a4d-c0a1-4868-a3ae-f4b0da2f92fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:13.482826 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28444a4d-c0a1-4868-a3ae-f4b0da2f92fb service neutron] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:13.557358 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 884/1784] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:26:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:26:13.705341 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66e6f82-656f-4b82-ae86-8b9e7ae4a3ad admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:13.718038 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e430a77-b7fa-48b2-8baa-9571a30a4e6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:13.718728 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e430a77-b7fa-48b2-8baa-9571a30a4e6a admin admin] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:13.719066 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e430a77-b7fa-48b2-8baa-9571a30a4e6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:13.719342 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e430a77-b7fa-48b2-8baa-9571a30a4e6a admin admin] PATH_INFO: `/v3/users` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:13.719982 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e430a77-b7fa-48b2-8baa-9571a30a4e6a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:13.724724 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e430a77-b7fa-48b2-8baa-9571a30a4e6a admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:13.795319 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28444a4d-c0a1-4868-a3ae-f4b0da2f92fb service neutron] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:13.799771 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-28444a4d-c0a1-4868-a3ae-f4b0da2f92fb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:26:13.833782 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28444a4d-c0a1-4868-a3ae-f4b0da2f92fb service neutron] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:13.848356 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986fc89b-2dba-4a1e-a15a-9c3126b30771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:13.850611 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986fc89b-2dba-4a1e-a15a-9c3126b30771 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:13.850611 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986fc89b-2dba-4a1e-a15a-9c3126b30771 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:13.850611 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986fc89b-2dba-4a1e-a15a-9c3126b30771 admin admin] PATH_INFO: `/v3/users/8bfc35543ede430fbe61009be0eadfbb` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:13.856144 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986fc89b-2dba-4a1e-a15a-9c3126b30771 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8bfc35543ede430fbe61009be0eadfbb)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:13.860647 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986fc89b-2dba-4a1e-a15a-9c3126b30771 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:13.877222 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 901/1785] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 24 17:26:13 2022] POST /identity/v3/users => generated 292 bytes in 457 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:26:13.881756 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e430a77-b7fa-48b2-8baa-9571a30a4e6a admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:13.913403 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e430a77-b7fa-48b2-8baa-9571a30a4e6a admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:13.935798 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd11de69-b8af-4af5-92f7-b0c8104466ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:13.937574 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd11de69-b8af-4af5-92f7-b0c8104466ea admin admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:13.937962 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd11de69-b8af-4af5-92f7-b0c8104466ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:13.938318 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd11de69-b8af-4af5-92f7-b0c8104466ea admin admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:13.941464 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd11de69-b8af-4af5-92f7-b0c8104466ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:13.947696 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd11de69-b8af-4af5-92f7-b0c8104466ea admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:13.969102 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 902/1786] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:26:13 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:26:13.973510 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd11de69-b8af-4af5-92f7-b0c8104466ea admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:13.986271 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd11de69-b8af-4af5-92f7-b0c8104466ea admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:13.996366 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ed5243-05d5-4493-a205-efa3ec5c93ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:13.998376 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ed5243-05d5-4493-a205-efa3ec5c93ea admin admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:13.999652 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ed5243-05d5-4493-a205-efa3ec5c93ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:14.000336 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ed5243-05d5-4493-a205-efa3ec5c93ea admin admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:14.002381 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ed5243-05d5-4493-a205-efa3ec5c93ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:14.004401 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ed5243-05d5-4493-a205-efa3ec5c93ea admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:14.023870 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 903/1787] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:26:13 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:26:14.029434 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ed5243-05d5-4493-a205-efa3ec5c93ea admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:14.061656 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ed5243-05d5-4493-a205-efa3ec5c93ea admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:14.070807 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed4dc4a-9413-4e79-852e-91460e38a568 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:14.089161 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed4dc4a-9413-4e79-852e-91460e38a568 admin admin] REQUEST_METHOD: `PUT` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:14.089161 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed4dc4a-9413-4e79-852e-91460e38a568 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:14.089161 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed4dc4a-9413-4e79-852e-91460e38a568 admin admin] PATH_INFO: `/v3/projects/97fcab3e3f6d45daa970b89b51b7d109/users/b5c65900d2e64e96b699024e86474d72/roles/60dc7e16aee746d5a3e8aa27d7d8525c` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:14.130020 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-986fc89b-2dba-4a1e-a15a-9c3126b30771 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8bfc35543ede430fbe61009be0eadfbb'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:14.132461 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed4dc4a-9413-4e79-852e-91460e38a568 admin admin] RBAC: Authorizing `identity:create_grant(project_id=97fcab3e3f6d45daa970b89b51b7d109, user_id=b5c65900d2e64e96b699024e86474d72, role_id=60dc7e16aee746d5a3e8aa27d7d8525c)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:14.141530 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed4dc4a-9413-4e79-852e-91460e38a568 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:14.249229 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 904/1788] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 24 17:26:14 2022] PUT /identity/v3/projects/97fcab3e3f6d45daa970b89b51b7d109/users/b5c65900d2e64e96b699024e86474d72/roles/60dc7e16aee746d5a3e8aa27d7d8525c => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:26:14.256727 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-986fc89b-2dba-4a1e-a15a-9c3126b30771 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8bfc35543ede430fbe61009be0eadfbb'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:14.264906 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-986fc89b-2dba-4a1e-a15a-9c3126b30771 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8bfc35543ede430fbe61009be0eadfbb'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:14.271468 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a2abd0-89da-4bc4-833b-7c0e56f618ad None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:14.271468 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a2abd0-89da-4bc4-833b-7c0e56f618ad None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:14.271468 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a2abd0-89da-4bc4-833b-7c0e56f618ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:14.286311 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-986fc89b-2dba-4a1e-a15a-9c3126b30771 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8bfc35543ede430fbe61009be0eadfbb'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:14.293469 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-986fc89b-2dba-4a1e-a15a-9c3126b30771 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8bfc35543ede430fbe61009be0eadfbb'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:14.371025 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 885/1789] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 24 17:26:13 2022] DELETE /identity/v3/users/8bfc35543ede430fbe61009be0eadfbb => generated 0 bytes in 577 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:26:14.375724 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824e4b41-3b73-4955-81e6-367abe1e02fb None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:14.376410 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824e4b41-3b73-4955-81e6-367abe1e02fb None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:14.376707 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-37a2abd0-89da-4bc4-833b-7c0e56f618ad None None] MFA Rules not processed for user `b5c65900d2e64e96b699024e86474d72`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:26:14.377471 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824e4b41-3b73-4955-81e6-367abe1e02fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:14.381219 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-37a2abd0-89da-4bc4-833b-7c0e56f618ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:26:14.453587 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-824e4b41-3b73-4955-81e6-367abe1e02fb None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:26:14.457706 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-824e4b41-3b73-4955-81e6-367abe1e02fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:26:14.610729 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 905/1790] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:26:14 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 348 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:26:14.614896 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a2abd0-89da-4bc4-833b-7c0e56f618ad None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:14.618423 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-37a2abd0-89da-4bc4-833b-7c0e56f618ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:26:14.641469 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 886/1791] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:26:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:26:14.693006 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:26:14.774418 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a2abd0-89da-4bc4-833b-7c0e56f618ad None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:14.780606 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824e4b41-3b73-4955-81e6-367abe1e02fb None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:14.783570 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-824e4b41-3b73-4955-81e6-367abe1e02fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:26:14.794654 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:26:14.800983 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6b3d3d-ee40-4c6b-8b6c-98735145729a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:14.801657 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6b3d3d-ee40-4c6b-8b6c-98735145729a admin admin] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:14.801939 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6b3d3d-ee40-4c6b-8b6c-98735145729a admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:14.802212 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6b3d3d-ee40-4c6b-8b6c-98735145729a admin admin] PATH_INFO: `/v3/projects` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:14.802863 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6b3d3d-ee40-4c6b-8b6c-98735145729a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:14.811729 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6b3d3d-ee40-4c6b-8b6c-98735145729a admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:14.939203 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 906/1792] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 24 17:26:14 2022] POST /identity/v3/projects => generated 385 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:26:14.940458 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6b3d3d-ee40-4c6b-8b6c-98735145729a admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:15.022016 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824e4b41-3b73-4955-81e6-367abe1e02fb None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:15.215578 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6b3d3d-ee40-4c6b-8b6c-98735145729a admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:15.222789 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e8a0f2-27ff-482c-b15a-05f16413d64f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:15.230305 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e8a0f2-27ff-482c-b15a-05f16413d64f admin admin] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:15.230932 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e8a0f2-27ff-482c-b15a-05f16413d64f admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:15.233833 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e8a0f2-27ff-482c-b15a-05f16413d64f admin admin] PATH_INFO: `/v3/users` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:15.234175 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e8a0f2-27ff-482c-b15a-05f16413d64f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:15.236389 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e8a0f2-27ff-482c-b15a-05f16413d64f admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:15.267784 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a86bcb-9b57-49a2-9c3f-28b8b3a0113c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e9ed4d47aef4140bfb2d3bb798cc559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:15.270365 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a86bcb-9b57-49a2-9c3f-28b8b3a0113c service neutron] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:15.270717 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a86bcb-9b57-49a2-9c3f-28b8b3a0113c service neutron] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:15.271051 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a86bcb-9b57-49a2-9c3f-28b8b3a0113c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:15.291498 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a86bcb-9b57-49a2-9c3f-28b8b3a0113c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:15.294172 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a86bcb-9b57-49a2-9c3f-28b8b3a0113c service neutron] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:15.323694 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 887/1793] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:26:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 544 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:26:15.380129 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 907/1794] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 24 17:26:14 2022] POST /identity/v3/users => generated 299 bytes in 440 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:26:15.383859 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a86bcb-9b57-49a2-9c3f-28b8b3a0113c service neutron] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:15.408210 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a86bcb-9b57-49a2-9c3f-28b8b3a0113c service neutron] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:15.413939 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15173aef-f896-453e-b1d1-151328c81686 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:15.423236 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15173aef-f896-453e-b1d1-151328c81686 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:15.423236 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15173aef-f896-453e-b1d1-151328c81686 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:15.423236 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15173aef-f896-453e-b1d1-151328c81686 admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:15.423236 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15173aef-f896-453e-b1d1-151328c81686 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:15.423236 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15173aef-f896-453e-b1d1-151328c81686 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:15.436529 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 888/1795] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:26:15 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:26:15.441466 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e8a0f2-27ff-482c-b15a-05f16413d64f admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:15.457232 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e8a0f2-27ff-482c-b15a-05f16413d64f admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:15.469183 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b52916-de1f-4220-96ae-cd07d2bd3fb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:15.470498 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b52916-de1f-4220-96ae-cd07d2bd3fb8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:15.470787 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b52916-de1f-4220-96ae-cd07d2bd3fb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:15.471196 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b52916-de1f-4220-96ae-cd07d2bd3fb8 admin admin] PATH_INFO: `/v3/projects/059cd28fa7d74054bc807c2b7974073a/users/d7cf12946e364d318ef67f5276e3ff96/roles/c5346bc54abf445f9900edf94264c850` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:15.503656 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b52916-de1f-4220-96ae-cd07d2bd3fb8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=059cd28fa7d74054bc807c2b7974073a, user_id=d7cf12946e364d318ef67f5276e3ff96, role_id=c5346bc54abf445f9900edf94264c850)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:15.509618 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b52916-de1f-4220-96ae-cd07d2bd3fb8 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:15.655745 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15173aef-f896-453e-b1d1-151328c81686 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:15.656403 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 908/1796] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 24 17:26:15 2022] PUT /identity/v3/projects/059cd28fa7d74054bc807c2b7974073a/users/d7cf12946e364d318ef67f5276e3ff96/roles/c5346bc54abf445f9900edf94264c850 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:26:15.673017 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15173aef-f896-453e-b1d1-151328c81686 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:15.764127 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:26:15.774887 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1673c8-c0c1-406a-bb04-e76afed9468a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:15.775817 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1673c8-c0c1-406a-bb04-e76afed9468a admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:15.776267 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1673c8-c0c1-406a-bb04-e76afed9468a admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:15.776546 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1673c8-c0c1-406a-bb04-e76afed9468a admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:15.777241 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1673c8-c0c1-406a-bb04-e76afed9468a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:15.779072 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1673c8-c0c1-406a-bb04-e76afed9468a admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:15.795998 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 889/1797] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:26:15 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:26:15.800771 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b52916-de1f-4220-96ae-cd07d2bd3fb8 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:15.821138 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b52916-de1f-4220-96ae-cd07d2bd3fb8 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:15.837737 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff818f7b-fe78-4526-98c7-a462900438a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:15.845166 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:26:15.846102 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff818f7b-fe78-4526-98c7-a462900438a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:15.846416 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff818f7b-fe78-4526-98c7-a462900438a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:15.846694 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff818f7b-fe78-4526-98c7-a462900438a8 admin admin] PATH_INFO: `/v3/projects/059cd28fa7d74054bc807c2b7974073a/users/d7cf12946e364d318ef67f5276e3ff96/roles/c5346bc54abf445f9900edf94264c850` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:15.851814 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff818f7b-fe78-4526-98c7-a462900438a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=059cd28fa7d74054bc807c2b7974073a, user_id=d7cf12946e364d318ef67f5276e3ff96, role_id=c5346bc54abf445f9900edf94264c850)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:15.864853 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff818f7b-fe78-4526-98c7-a462900438a8 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:15.884294 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 909/1798] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 24 17:26:15 2022] PUT /identity/v3/projects/059cd28fa7d74054bc807c2b7974073a/users/d7cf12946e364d318ef67f5276e3ff96/roles/c5346bc54abf445f9900edf94264c850 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:26:15.887461 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaadd2b-fb9f-4521-92f8-51985ed3e7d4 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:15.888839 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaadd2b-fb9f-4521-92f8-51985ed3e7d4 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:15.888839 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaadd2b-fb9f-4521-92f8-51985ed3e7d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:15.954163 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-fcaadd2b-fb9f-4521-92f8-51985ed3e7d4 None None] MFA Rules not processed for user `d7cf12946e364d318ef67f5276e3ff96`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:26:15.956321 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-fcaadd2b-fb9f-4521-92f8-51985ed3e7d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:26:16.086689 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:26:16.222444 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 890/1799] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:26:15 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 336 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:26:16.249802 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff818f7b-fe78-4526-98c7-a462900438a8 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:16.253266 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-ff818f7b-fe78-4526-98c7-a462900438a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:26:16.368542 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff818f7b-fe78-4526-98c7-a462900438a8 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:16.378774 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a8681a-4b8b-4149-9d7b-89d42160e7e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be06246d47584b529150ceb98a5cfd95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:16.379892 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a8681a-4b8b-4149-9d7b-89d42160e7e6 service cinder] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:16.381057 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a8681a-4b8b-4149-9d7b-89d42160e7e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:16.382048 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a8681a-4b8b-4149-9d7b-89d42160e7e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:16.397640 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82471000-1661-47fa-ac67-0a539a44449e None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:16.397992 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82471000-1661-47fa-ac67-0a539a44449e None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:16.398269 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82471000-1661-47fa-ac67-0a539a44449e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:16.447733 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-82471000-1661-47fa-ac67-0a539a44449e None None] MFA Rules not processed for user `c379c845fccd43c08e8699976d506801`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:26:16.449538 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-82471000-1661-47fa-ac67-0a539a44449e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:26:16.572112 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a8681a-4b8b-4149-9d7b-89d42160e7e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:16.574696 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a8681a-4b8b-4149-9d7b-89d42160e7e6 service cinder] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:16.587208 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 910/1800] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:26:16 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:26:16.606574 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 891/1801] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:26:16 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:26:16.751088 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:DELETE=2,keystone:SELECT=98 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:26:16.970022 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:26:17.444051 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:26:17.704685 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a8681a-4b8b-4149-9d7b-89d42160e7e6 service cinder] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:17.706548 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-47a8681a-4b8b-4149-9d7b-89d42160e7e6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:26:17.846049 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a8681a-4b8b-4149-9d7b-89d42160e7e6 service cinder] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:17.852657 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70c4e3b-45c8-4bc3-9637-77b884478d73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:17.853556 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70c4e3b-45c8-4bc3-9637-77b884478d73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:17.853981 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70c4e3b-45c8-4bc3-9637-77b884478d73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:17.854423 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70c4e3b-45c8-4bc3-9637-77b884478d73 admin admin] PATH_INFO: `/v3/projects/aba808fdeca64e9685a823ba97626f26` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:17.856312 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70c4e3b-45c8-4bc3-9637-77b884478d73 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aba808fdeca64e9685a823ba97626f26)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:17.858538 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70c4e3b-45c8-4bc3-9637-77b884478d73 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:17.973208 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0b638c-5465-4783-9435-c783bb9ecb3f None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:17.976428 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0b638c-5465-4783-9435-c783bb9ecb3f None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:17.976428 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0b638c-5465-4783-9435-c783bb9ecb3f None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:17.976428 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 892/1802] 10.222.0.18 () {54 vars in 921 bytes} [Wed Aug 24 17:26:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:26:18.001333 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe9fdae-a82d-4f96-b5a6-afa9fa496ad3 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:18.005016 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe9fdae-a82d-4f96-b5a6-afa9fa496ad3 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:18.005016 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe9fdae-a82d-4f96-b5a6-afa9fa496ad3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:18.038845 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-3fe9fdae-a82d-4f96-b5a6-afa9fa496ad3 None None] MFA Rules not processed for user `557aca9255cb4bbb8af52e796c5c5177`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:26:18.040429 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-3fe9fdae-a82d-4f96-b5a6-afa9fa496ad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:26:18.060138 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-e70c4e3b-45c8-4bc3-9637-77b884478d73 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aba808fdeca64e9685a823ba97626f26', 'request_id': 'req-e70c4e3b-45c8-4bc3-9637-77b884478d73'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:18.177992 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-e70c4e3b-45c8-4bc3-9637-77b884478d73 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aba808fdeca64e9685a823ba97626f26', 'request_id': 'req-e70c4e3b-45c8-4bc3-9637-77b884478d73'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:18.188270 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-e70c4e3b-45c8-4bc3-9637-77b884478d73 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aba808fdeca64e9685a823ba97626f26', 'request_id': 'req-e70c4e3b-45c8-4bc3-9637-77b884478d73'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:18.239395 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-e70c4e3b-45c8-4bc3-9637-77b884478d73 admin admin] The token cache is being invalidate because project aba808fdeca64e9685a823ba97626f26 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81731) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:26:18.242195 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-e70c4e3b-45c8-4bc3-9637-77b884478d73 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:18.244371 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-e70c4e3b-45c8-4bc3-9637-77b884478d73 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:18.254809 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 911/1803] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 24 17:26:17 2022] DELETE /identity/v3/projects/aba808fdeca64e9685a823ba97626f26 => generated 0 bytes in 547 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:26:18.255911 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70c4e3b-45c8-4bc3-9637-77b884478d73 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:18.263132 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-e70c4e3b-45c8-4bc3-9637-77b884478d73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:26:18.333190 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 893/1804] 10.222.0.18 () {58 vars in 1018 bytes} [Wed Aug 24 17:26:17 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 333 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:26:18.404615 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70c4e3b-45c8-4bc3-9637-77b884478d73 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:18.409423 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7029dae9-5676-4b44-983e-76851cf55777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:18.410090 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7029dae9-5676-4b44-983e-76851cf55777 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:18.410373 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7029dae9-5676-4b44-983e-76851cf55777 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:18.410651 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7029dae9-5676-4b44-983e-76851cf55777 admin admin] PATH_INFO: `/v3/users/780a0cd6a12a493ca52183d2740ca488` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:18.411827 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7029dae9-5676-4b44-983e-76851cf55777 admin admin] RBAC: Authorizing `identity:delete_user(user_id=780a0cd6a12a493ca52183d2740ca488)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:18.414610 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7029dae9-5676-4b44-983e-76851cf55777 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:18.738352 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-7029dae9-5676-4b44-983e-76851cf55777 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '780a0cd6a12a493ca52183d2740ca488'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:18.762357 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe9fdae-a82d-4f96-b5a6-afa9fa496ad3 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:18.764815 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-3fe9fdae-a82d-4f96-b5a6-afa9fa496ad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:26:18.791280 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-7029dae9-5676-4b44-983e-76851cf55777 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '780a0cd6a12a493ca52183d2740ca488'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:18.804687 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe9fdae-a82d-4f96-b5a6-afa9fa496ad3 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:18.817938 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c362a1f0-9c45-4e31-86ac-05ed78cd7ba9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:18.825616 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c362a1f0-9c45-4e31-86ac-05ed78cd7ba9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:18.826037 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c362a1f0-9c45-4e31-86ac-05ed78cd7ba9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:18.826691 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c362a1f0-9c45-4e31-86ac-05ed78cd7ba9 admin admin] PATH_INFO: `/v3/projects/f96d1b72ed764675a060748e8e9f5791` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:18.838786 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c362a1f0-9c45-4e31-86ac-05ed78cd7ba9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f96d1b72ed764675a060748e8e9f5791)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:18.843678 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c362a1f0-9c45-4e31-86ac-05ed78cd7ba9 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:18.889998 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-7029dae9-5676-4b44-983e-76851cf55777 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '780a0cd6a12a493ca52183d2740ca488'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:18.891333 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-7029dae9-5676-4b44-983e-76851cf55777 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '780a0cd6a12a493ca52183d2740ca488'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:18.904298 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-7029dae9-5676-4b44-983e-76851cf55777 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '780a0cd6a12a493ca52183d2740ca488'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:18.922950 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 912/1805] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 24 17:26:18 2022] DELETE /identity/v3/users/780a0cd6a12a493ca52183d2740ca488 => generated 0 bytes in 668 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:26:19.088905 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-c362a1f0-9c45-4e31-86ac-05ed78cd7ba9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f96d1b72ed764675a060748e8e9f5791', 'request_id': 'req-c362a1f0-9c45-4e31-86ac-05ed78cd7ba9'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:19.226652 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-c362a1f0-9c45-4e31-86ac-05ed78cd7ba9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f96d1b72ed764675a060748e8e9f5791', 'request_id': 'req-c362a1f0-9c45-4e31-86ac-05ed78cd7ba9'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:19.229305 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:26:19.268027 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-c362a1f0-9c45-4e31-86ac-05ed78cd7ba9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f96d1b72ed764675a060748e8e9f5791', 'request_id': 'req-c362a1f0-9c45-4e31-86ac-05ed78cd7ba9'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:19.313261 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-c362a1f0-9c45-4e31-86ac-05ed78cd7ba9 admin admin] The token cache is being invalidate because project f96d1b72ed764675a060748e8e9f5791 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81732) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:26:19.313816 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-c362a1f0-9c45-4e31-86ac-05ed78cd7ba9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:19.315978 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-c362a1f0-9c45-4e31-86ac-05ed78cd7ba9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:19.325273 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 894/1806] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 24 17:26:18 2022] DELETE /identity/v3/projects/f96d1b72ed764675a060748e8e9f5791 => generated 0 bytes in 565 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:26:19.331727 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c362a1f0-9c45-4e31-86ac-05ed78cd7ba9 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:19.331931 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8bdf9e-a269-4a8c-8c7f-4791c4578cbf None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:19.334568 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8bdf9e-a269-4a8c-8c7f-4791c4578cbf None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:19.335349 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8bdf9e-a269-4a8c-8c7f-4791c4578cbf None None] PATH_INFO: `/` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:19.342055 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 913/1807] 10.222.0.18 () {54 vars in 921 bytes} [Wed Aug 24 17:26:19 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:26:19.342276 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-c362a1f0-9c45-4e31-86ac-05ed78cd7ba9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:26:19.483769 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152f7111-a646-47d9-b9f8-72f963f6843c None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:19.484665 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152f7111-a646-47d9-b9f8-72f963f6843c None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:19.484814 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152f7111-a646-47d9-b9f8-72f963f6843c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:19.575561 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-152f7111-a646-47d9-b9f8-72f963f6843c None None] MFA Rules not processed for user `557aca9255cb4bbb8af52e796c5c5177`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:26:19.576809 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-152f7111-a646-47d9-b9f8-72f963f6843c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:26:19.585423 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c362a1f0-9c45-4e31-86ac-05ed78cd7ba9 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:19.623623 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba41156-68b1-4a2b-8361-44bfe5a92e37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:19.624923 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba41156-68b1-4a2b-8361-44bfe5a92e37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:19.625112 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba41156-68b1-4a2b-8361-44bfe5a92e37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:19.625338 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba41156-68b1-4a2b-8361-44bfe5a92e37 admin admin] PATH_INFO: `/v3/users/d8be0791e2b740dfba62612d5a64247f` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:19.628022 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba41156-68b1-4a2b-8361-44bfe5a92e37 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d8be0791e2b740dfba62612d5a64247f)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:19.633878 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba41156-68b1-4a2b-8361-44bfe5a92e37 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:19.830593 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 914/1808] 10.222.0.18 () {58 vars in 1018 bytes} [Wed Aug 24 17:26:19 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 378 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:26:19.832939 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152f7111-a646-47d9-b9f8-72f963f6843c None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:19.835043 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-152f7111-a646-47d9-b9f8-72f963f6843c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:26:19.869013 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152f7111-a646-47d9-b9f8-72f963f6843c None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:19.881913 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6ba432-4a93-480b-9ee2-15355221c752 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557aca9255cb4bbb8af52e796c5c5177', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:19.882713 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6ba432-4a93-480b-9ee2-15355221c752 service nova] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:19.883204 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6ba432-4a93-480b-9ee2-15355221c752 service nova] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:19.883523 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6ba432-4a93-480b-9ee2-15355221c752 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:19.885593 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-9e6ba432-4a93-480b-9ee2-15355221c752 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:26:19.903592 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6ba432-4a93-480b-9ee2-15355221c752 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:19.906273 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6ba432-4a93-480b-9ee2-15355221c752 service nova] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:19.920736 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 915/1809] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:26:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:26:19.922333 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aebb7a-1fe8-4902-bf95-7629ecd86c3e None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:19.922856 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aebb7a-1fe8-4902-bf95-7629ecd86c3e None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:19.923424 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aebb7a-1fe8-4902-bf95-7629ecd86c3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:19.930757 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:26:19.960218 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-4ba41156-68b1-4a2b-8361-44bfe5a92e37 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd8be0791e2b740dfba62612d5a64247f'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:19.986735 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-64aebb7a-1fe8-4902-bf95-7629ecd86c3e None None] MFA Rules not processed for user `160ce86c89ea449da346d98d20ce2a9e`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:26:19.988435 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-64aebb7a-1fe8-4902-bf95-7629ecd86c3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:26:20.071832 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:26:20.095298 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-4ba41156-68b1-4a2b-8361-44bfe5a92e37 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd8be0791e2b740dfba62612d5a64247f'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:20.096087 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-4ba41156-68b1-4a2b-8361-44bfe5a92e37 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd8be0791e2b740dfba62612d5a64247f'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:20.111540 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-4ba41156-68b1-4a2b-8361-44bfe5a92e37 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd8be0791e2b740dfba62612d5a64247f'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:20.113026 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-4ba41156-68b1-4a2b-8361-44bfe5a92e37 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd8be0791e2b740dfba62612d5a64247f'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:20.134619 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 916/1810] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:26:19 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:26:20.164323 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64aebb7a-1fe8-4902-bf95-7629ecd86c3e None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:20.175557 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-64aebb7a-1fe8-4902-bf95-7629ecd86c3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:26:20.196248 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 895/1811] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 24 17:26:19 2022] DELETE /identity/v3/users/d8be0791e2b740dfba62612d5a64247f => generated 0 bytes in 866 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:26:20.216831 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64aebb7a-1fe8-4902-bf95-7629ecd86c3e None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:20.394168 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa72ec7-f106-44ea-a8e2-23de145830a2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557aca9255cb4bbb8af52e796c5c5177', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:20.394935 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa72ec7-f106-44ea-a8e2-23de145830a2 service nova] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:20.395330 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa72ec7-f106-44ea-a8e2-23de145830a2 service nova] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:20.395592 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa72ec7-f106-44ea-a8e2-23de145830a2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:20.400302 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-eaa72ec7-f106-44ea-a8e2-23de145830a2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:26:20.424634 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa72ec7-f106-44ea-a8e2-23de145830a2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:20.434995 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa72ec7-f106-44ea-a8e2-23de145830a2 service nova] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:20.448969 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 917/1812] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:26:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:26:20.536139 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:26:20.969274 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:26:21.463488 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba41156-68b1-4a2b-8361-44bfe5a92e37 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:21.465487 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-4ba41156-68b1-4a2b-8361-44bfe5a92e37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:26:21.540085 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:26:21.620796 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba41156-68b1-4a2b-8361-44bfe5a92e37 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:21.657281 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b323f2-d0b2-4db0-9fa5-06ca4510298f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:21.658609 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b323f2-d0b2-4db0-9fa5-06ca4510298f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:21.661097 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b323f2-d0b2-4db0-9fa5-06ca4510298f admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:21.661444 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b323f2-d0b2-4db0-9fa5-06ca4510298f admin admin] PATH_INFO: `/v3/projects/7d7968fbdd3c44af8c4b12d9b0cfa5a9` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:21.671289 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b323f2-d0b2-4db0-9fa5-06ca4510298f admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d7968fbdd3c44af8c4b12d9b0cfa5a9)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:21.673822 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b323f2-d0b2-4db0-9fa5-06ca4510298f admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:21.956839 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-26b323f2-d0b2-4db0-9fa5-06ca4510298f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d7968fbdd3c44af8c4b12d9b0cfa5a9', 'request_id': 'req-26b323f2-d0b2-4db0-9fa5-06ca4510298f'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:22.077967 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-26b323f2-d0b2-4db0-9fa5-06ca4510298f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d7968fbdd3c44af8c4b12d9b0cfa5a9', 'request_id': 'req-26b323f2-d0b2-4db0-9fa5-06ca4510298f'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:22.086886 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-26b323f2-d0b2-4db0-9fa5-06ca4510298f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d7968fbdd3c44af8c4b12d9b0cfa5a9', 'request_id': 'req-26b323f2-d0b2-4db0-9fa5-06ca4510298f'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:22.105349 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-26b323f2-d0b2-4db0-9fa5-06ca4510298f admin admin] The token cache is being invalidate because project 7d7968fbdd3c44af8c4b12d9b0cfa5a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81732) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:26:22.105974 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-26b323f2-d0b2-4db0-9fa5-06ca4510298f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:22.107684 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-26b323f2-d0b2-4db0-9fa5-06ca4510298f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:22.110255 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 896/1813] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 24 17:26:21 2022] DELETE /identity/v3/projects/7d7968fbdd3c44af8c4b12d9b0cfa5a9 => generated 0 bytes in 647 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:26:22.117350 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60835d7a-ee1c-43b7-bcc0-e1f2fbb31093 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:22.118462 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60835d7a-ee1c-43b7-bcc0-e1f2fbb31093 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:22.119019 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60835d7a-ee1c-43b7-bcc0-e1f2fbb31093 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:22.158180 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-60835d7a-ee1c-43b7-bcc0-e1f2fbb31093 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:26:22.165603 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-60835d7a-ee1c-43b7-bcc0-e1f2fbb31093 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:26:22.310519 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 918/1814] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:26:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:26:22.319603 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78547f8b-a774-461b-833f-31d8b23116a1 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:22.323272 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78547f8b-a774-461b-833f-31d8b23116a1 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:22.323877 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78547f8b-a774-461b-833f-31d8b23116a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:22.372229 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-78547f8b-a774-461b-833f-31d8b23116a1 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:26:22.374253 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-78547f8b-a774-461b-833f-31d8b23116a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:26:22.389934 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 897/1815] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:26:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:26:22.391132 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60835d7a-ee1c-43b7-bcc0-e1f2fbb31093 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:22.405199 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60835d7a-ee1c-43b7-bcc0-e1f2fbb31093 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:22.411810 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b10868-26d7-4df6-b299-f406eac81c58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:22.412806 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b10868-26d7-4df6-b299-f406eac81c58 admin admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:22.413371 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b10868-26d7-4df6-b299-f406eac81c58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:22.413953 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b10868-26d7-4df6-b299-f406eac81c58 admin admin] PATH_INFO: `/v3/domains` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:22.414711 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b10868-26d7-4df6-b299-f406eac81c58 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:26:22.415035 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b10868-26d7-4df6-b299-f406eac81c58 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:22.418091 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b10868-26d7-4df6-b299-f406eac81c58 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:22.453181 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78547f8b-a774-461b-833f-31d8b23116a1 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:22.459940 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 919/1816] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 24 17:26:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:26:22.465893 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78547f8b-a774-461b-833f-31d8b23116a1 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:22.473395 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d163a73-56dc-46ca-b957-bdc86792bb30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:22.474617 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d163a73-56dc-46ca-b957-bdc86792bb30 admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:22.475563 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d163a73-56dc-46ca-b957-bdc86792bb30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:22.476097 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d163a73-56dc-46ca-b957-bdc86792bb30 admin admin] PATH_INFO: `/v3/projects` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:22.478043 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d163a73-56dc-46ca-b957-bdc86792bb30 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:22.481318 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d163a73-56dc-46ca-b957-bdc86792bb30 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:22.622032 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 898/1817] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 24 17:26:22 2022] POST /identity/v3/projects => generated 373 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:26:22.622992 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b10868-26d7-4df6-b299-f406eac81c58 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:22.634244 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b10868-26d7-4df6-b299-f406eac81c58 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:22.720412 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd346fa-a3ca-4974-80f0-49e75ceb6092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:22.721032 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd346fa-a3ca-4974-80f0-49e75ceb6092 admin admin] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:22.721321 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd346fa-a3ca-4974-80f0-49e75ceb6092 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:22.721605 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd346fa-a3ca-4974-80f0-49e75ceb6092 admin admin] PATH_INFO: `/v3/users` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:22.722227 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd346fa-a3ca-4974-80f0-49e75ceb6092 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:22.724503 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd346fa-a3ca-4974-80f0-49e75ceb6092 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:22.846079 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d163a73-56dc-46ca-b957-bdc86792bb30 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:22.848039 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-2d163a73-56dc-46ca-b957-bdc86792bb30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:26:22.864484 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 920/1818] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 24 17:26:22 2022] POST /identity/v3/users => generated 287 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:26:22.883350 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd346fa-a3ca-4974-80f0-49e75ceb6092 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:22.884573 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d163a73-56dc-46ca-b957-bdc86792bb30 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:22.915498 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd346fa-a3ca-4974-80f0-49e75ceb6092 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:22.917695 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6432ad7-9fd8-46c6-96ee-946da91853f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:22.918535 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6432ad7-9fd8-46c6-96ee-946da91853f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:22.918978 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6432ad7-9fd8-46c6-96ee-946da91853f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:22.919407 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6432ad7-9fd8-46c6-96ee-946da91853f4 admin admin] PATH_INFO: `/v3/projects/6f663c2438eb48a5a861ae6d9524be47` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:22.923904 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6432ad7-9fd8-46c6-96ee-946da91853f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f663c2438eb48a5a861ae6d9524be47)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:22.930623 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6432ad7-9fd8-46c6-96ee-946da91853f4 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:22.931898 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104ee11d-ab1d-4ed0-b4c6-b415b15f0e3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:22.932650 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104ee11d-ab1d-4ed0-b4c6-b415b15f0e3f admin admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:22.932989 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104ee11d-ab1d-4ed0-b4c6-b415b15f0e3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:22.938315 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104ee11d-ab1d-4ed0-b4c6-b415b15f0e3f admin admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:22.938315 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104ee11d-ab1d-4ed0-b4c6-b415b15f0e3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:22.938315 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104ee11d-ab1d-4ed0-b4c6-b415b15f0e3f admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:22.973743 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 921/1819] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:26:22 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:26:22.976646 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104ee11d-ab1d-4ed0-b4c6-b415b15f0e3f admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:23.009093 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104ee11d-ab1d-4ed0-b4c6-b415b15f0e3f admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:23.016199 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955c52f9-acae-440f-a139-c08706f23337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:23.017103 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955c52f9-acae-440f-a139-c08706f23337 admin admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:23.017532 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955c52f9-acae-440f-a139-c08706f23337 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:23.017936 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955c52f9-acae-440f-a139-c08706f23337 admin admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:23.018827 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955c52f9-acae-440f-a139-c08706f23337 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:23.020715 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955c52f9-acae-440f-a139-c08706f23337 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:23.043651 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 922/1820] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:26:22 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:26:23.071541 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955c52f9-acae-440f-a139-c08706f23337 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:23.096515 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955c52f9-acae-440f-a139-c08706f23337 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:23.124844 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3d0617-6590-414f-9735-b9a1017ee9ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:23.126262 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3d0617-6590-414f-9735-b9a1017ee9ea admin admin] REQUEST_METHOD: `PUT` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:23.126576 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3d0617-6590-414f-9735-b9a1017ee9ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:23.126884 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3d0617-6590-414f-9735-b9a1017ee9ea admin admin] PATH_INFO: `/v3/projects/bb4465ad66dc4fdca7943fd07cb82807/users/427a581c9c47411db485028e40830208/roles/60dc7e16aee746d5a3e8aa27d7d8525c` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:23.162540 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3d0617-6590-414f-9735-b9a1017ee9ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb4465ad66dc4fdca7943fd07cb82807, user_id=427a581c9c47411db485028e40830208, role_id=60dc7e16aee746d5a3e8aa27d7d8525c)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:23.166375 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3d0617-6590-414f-9735-b9a1017ee9ea admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:23.175250 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-e6432ad7-9fd8-46c6-96ee-946da91853f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f663c2438eb48a5a861ae6d9524be47', 'request_id': 'req-e6432ad7-9fd8-46c6-96ee-946da91853f4'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:23.267037 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-e6432ad7-9fd8-46c6-96ee-946da91853f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f663c2438eb48a5a861ae6d9524be47', 'request_id': 'req-e6432ad7-9fd8-46c6-96ee-946da91853f4'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:23.288879 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-e6432ad7-9fd8-46c6-96ee-946da91853f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f663c2438eb48a5a861ae6d9524be47', 'request_id': 'req-e6432ad7-9fd8-46c6-96ee-946da91853f4'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:23.292276 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 923/1821] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 24 17:26:23 2022] PUT /identity/v3/projects/bb4465ad66dc4fdca7943fd07cb82807/users/427a581c9c47411db485028e40830208/roles/60dc7e16aee746d5a3e8aa27d7d8525c => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:26:23.292276 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82af5e33-f36c-466a-9a89-41f12a73b760 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:23.292276 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82af5e33-f36c-466a-9a89-41f12a73b760 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:23.292276 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82af5e33-f36c-466a-9a89-41f12a73b760 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:23.307554 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-e6432ad7-9fd8-46c6-96ee-946da91853f4 admin admin] The token cache is being invalidate because project 6f663c2438eb48a5a861ae6d9524be47 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81732) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:26:23.309209 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-e6432ad7-9fd8-46c6-96ee-946da91853f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:23.322570 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-e6432ad7-9fd8-46c6-96ee-946da91853f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:23.325447 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:26:23.327897 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 899/1822] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 24 17:26:22 2022] DELETE /identity/v3/projects/6f663c2438eb48a5a861ae6d9524be47 => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:26:23.333062 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dde6d7-2acc-4f2a-b9e3-ded1c19d29bb None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:23.333378 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dde6d7-2acc-4f2a-b9e3-ded1c19d29bb None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:23.336445 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dde6d7-2acc-4f2a-b9e3-ded1c19d29bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:23.390573 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-59dde6d7-2acc-4f2a-b9e3-ded1c19d29bb None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:26:23.411016 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-59dde6d7-2acc-4f2a-b9e3-ded1c19d29bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:26:23.428556 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-82af5e33-f36c-466a-9a89-41f12a73b760 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:26:23.435203 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-82af5e33-f36c-466a-9a89-41f12a73b760 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:26:23.526171 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 900/1823] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:26:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:26:23.535403 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83814fc-e601-4a8a-abcc-71fd4019ec31 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:23.536001 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83814fc-e601-4a8a-abcc-71fd4019ec31 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:23.536506 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83814fc-e601-4a8a-abcc-71fd4019ec31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:23.581778 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-f83814fc-e601-4a8a-abcc-71fd4019ec31 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:26:23.583312 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-f83814fc-e601-4a8a-abcc-71fd4019ec31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:26:23.601028 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 901/1824] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:26:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:26:23.607123 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83814fc-e601-4a8a-abcc-71fd4019ec31 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:23.615357 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:26:23.626894 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83814fc-e601-4a8a-abcc-71fd4019ec31 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:23.644754 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6050a4b6-b904-4e7c-8cb1-2529d28ea978 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:23.645815 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6050a4b6-b904-4e7c-8cb1-2529d28ea978 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:23.646165 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6050a4b6-b904-4e7c-8cb1-2529d28ea978 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:23.646502 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6050a4b6-b904-4e7c-8cb1-2529d28ea978 admin admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:23.653589 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6050a4b6-b904-4e7c-8cb1-2529d28ea978 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:26:23.653589 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6050a4b6-b904-4e7c-8cb1-2529d28ea978 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:23.653589 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6050a4b6-b904-4e7c-8cb1-2529d28ea978 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:23.659180 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 924/1825] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:26:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:26:23.672898 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 902/1826] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 24 17:26:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:26:23.677825 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6050a4b6-b904-4e7c-8cb1-2529d28ea978 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:23.683608 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82af5e33-f36c-466a-9a89-41f12a73b760 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:23.689355 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-82af5e33-f36c-466a-9a89-41f12a73b760 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:26:23.709905 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6050a4b6-b904-4e7c-8cb1-2529d28ea978 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:23.716336 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722e17b1-a423-465b-967b-e5e7b4993fb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:23.717247 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722e17b1-a423-465b-967b-e5e7b4993fb1 admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:23.717624 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722e17b1-a423-465b-967b-e5e7b4993fb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:23.718083 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722e17b1-a423-465b-967b-e5e7b4993fb1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:23.719043 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722e17b1-a423-465b-967b-e5e7b4993fb1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:23.721507 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722e17b1-a423-465b-967b-e5e7b4993fb1 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:23.819739 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82af5e33-f36c-466a-9a89-41f12a73b760 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:23.834146 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fd38b2-900d-4763-97bc-cee65ab83ced service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e9ed4d47aef4140bfb2d3bb798cc559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:23.835111 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fd38b2-900d-4763-97bc-cee65ab83ced service neutron] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:23.835475 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fd38b2-900d-4763-97bc-cee65ab83ced service neutron] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:23.835877 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fd38b2-900d-4763-97bc-cee65ab83ced service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:23.856678 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fd38b2-900d-4763-97bc-cee65ab83ced service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:23.858279 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fd38b2-900d-4763-97bc-cee65ab83ced service neutron] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:23.864073 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 903/1827] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 24 17:26:23 2022] POST /identity/v3/projects => generated 385 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:26:23.867870 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722e17b1-a423-465b-967b-e5e7b4993fb1 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:23.881219 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 925/1828] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:26:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:26:23.992917 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722e17b1-a423-465b-967b-e5e7b4993fb1 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:23.999046 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369d4bd5-9131-414f-a707-985ac5ef2caa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:24.002014 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369d4bd5-9131-414f-a707-985ac5ef2caa admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:24.002014 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369d4bd5-9131-414f-a707-985ac5ef2caa admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:24.002014 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369d4bd5-9131-414f-a707-985ac5ef2caa admin admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:24.002014 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369d4bd5-9131-414f-a707-985ac5ef2caa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:24.011327 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369d4bd5-9131-414f-a707-985ac5ef2caa admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:24.187652 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 904/1829] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 24 17:26:23 2022] POST /identity/v3/users => generated 293 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:26:24.192134 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fd38b2-900d-4763-97bc-cee65ab83ced service neutron] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:24.205437 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fd38b2-900d-4763-97bc-cee65ab83ced service neutron] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:24.211801 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a208eb2-8759-43b6-8e90-bc901b530493 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:24.212499 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a208eb2-8759-43b6-8e90-bc901b530493 admin admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:24.223106 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a208eb2-8759-43b6-8e90-bc901b530493 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:24.223421 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a208eb2-8759-43b6-8e90-bc901b530493 admin admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:24.224784 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a208eb2-8759-43b6-8e90-bc901b530493 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:24.229944 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a208eb2-8759-43b6-8e90-bc901b530493 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:24.278858 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 926/1830] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:26:24 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:26:24.282107 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369d4bd5-9131-414f-a707-985ac5ef2caa admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:24.308952 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369d4bd5-9131-414f-a707-985ac5ef2caa admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:24.318767 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e96699-497e-4589-a54f-565adb618975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:24.320885 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e96699-497e-4589-a54f-565adb618975 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:24.320885 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e96699-497e-4589-a54f-565adb618975 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:24.320885 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e96699-497e-4589-a54f-565adb618975 admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:24.320885 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e96699-497e-4589-a54f-565adb618975 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:24.325860 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e96699-497e-4589-a54f-565adb618975 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:24.339188 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 905/1831] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:26:24 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:26:24.343618 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a208eb2-8759-43b6-8e90-bc901b530493 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:24.363257 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a208eb2-8759-43b6-8e90-bc901b530493 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:24.380187 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f434e15a-8b16-460e-9333-948e04ac489d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:24.380187 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f434e15a-8b16-460e-9333-948e04ac489d admin admin] REQUEST_METHOD: `PUT` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:24.380187 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f434e15a-8b16-460e-9333-948e04ac489d admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:24.380187 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f434e15a-8b16-460e-9333-948e04ac489d admin admin] PATH_INFO: `/v3/projects/95268d18617f41f186d2e95c3d1445c4/users/f37e50db294945c68d26d24da695e5f8/roles/60dc7e16aee746d5a3e8aa27d7d8525c` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:24.441243 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f434e15a-8b16-460e-9333-948e04ac489d admin admin] RBAC: Authorizing `identity:create_grant(project_id=95268d18617f41f186d2e95c3d1445c4, user_id=f37e50db294945c68d26d24da695e5f8, role_id=60dc7e16aee746d5a3e8aa27d7d8525c)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:24.444875 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f434e15a-8b16-460e-9333-948e04ac489d admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:24.569050 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 927/1832] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 24 17:26:24 2022] PUT /identity/v3/projects/95268d18617f41f186d2e95c3d1445c4/users/f37e50db294945c68d26d24da695e5f8/roles/60dc7e16aee746d5a3e8aa27d7d8525c => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:26:24.571758 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9361e65c-c7c3-47f5-b8ef-a450da5ac12d None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:24.573303 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9361e65c-c7c3-47f5-b8ef-a450da5ac12d None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:24.575810 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9361e65c-c7c3-47f5-b8ef-a450da5ac12d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:24.636297 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-9361e65c-c7c3-47f5-b8ef-a450da5ac12d None None] MFA Rules not processed for user `f37e50db294945c68d26d24da695e5f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:26:24.637925 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-9361e65c-c7c3-47f5-b8ef-a450da5ac12d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:26:24.751714 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 906/1833] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:26:24 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:26:24.752334 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f434e15a-8b16-460e-9333-948e04ac489d admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:24.834069 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f434e15a-8b16-460e-9333-948e04ac489d admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:24.839041 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e124a1ef-6922-4e7f-ba46-77f442673f82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:24.840203 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e124a1ef-6922-4e7f-ba46-77f442673f82 admin admin] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:24.840483 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e124a1ef-6922-4e7f-ba46-77f442673f82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:24.840751 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e124a1ef-6922-4e7f-ba46-77f442673f82 admin admin] PATH_INFO: `/v3/projects` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:24.841392 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e124a1ef-6922-4e7f-ba46-77f442673f82 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:24.844145 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e124a1ef-6922-4e7f-ba46-77f442673f82 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:25.101297 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 928/1834] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 24 17:26:24 2022] POST /identity/v3/projects => generated 385 bytes in 347 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:26:25.102922 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9361e65c-c7c3-47f5-b8ef-a450da5ac12d None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:25.121039 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9361e65c-c7c3-47f5-b8ef-a450da5ac12d None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:25.219621 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c25fe92-51d5-4bd2-bcde-458fc0099016 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:25.221402 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c25fe92-51d5-4bd2-bcde-458fc0099016 admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:25.221910 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c25fe92-51d5-4bd2-bcde-458fc0099016 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:25.222422 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c25fe92-51d5-4bd2-bcde-458fc0099016 admin admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:25.223407 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c25fe92-51d5-4bd2-bcde-458fc0099016 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:25.227174 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c25fe92-51d5-4bd2-bcde-458fc0099016 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:25.364842 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 907/1835] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 24 17:26:25 2022] POST /identity/v3/users => generated 299 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:26:25.370470 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e124a1ef-6922-4e7f-ba46-77f442673f82 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:25.388063 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e124a1ef-6922-4e7f-ba46-77f442673f82 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:25.394527 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72286aa5-90f0-4cf5-b433-722b73921b65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:25.395368 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72286aa5-90f0-4cf5-b433-722b73921b65 admin admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:25.395478 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72286aa5-90f0-4cf5-b433-722b73921b65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:25.395796 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72286aa5-90f0-4cf5-b433-722b73921b65 admin admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:25.396445 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72286aa5-90f0-4cf5-b433-722b73921b65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:25.398112 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72286aa5-90f0-4cf5-b433-722b73921b65 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:25.410544 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 929/1836] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:26:25 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:26:25.414949 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c25fe92-51d5-4bd2-bcde-458fc0099016 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:25.426481 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c25fe92-51d5-4bd2-bcde-458fc0099016 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:25.431403 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cc6af2-f6fe-4d99-b45f-ea79d1c73453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:25.432086 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cc6af2-f6fe-4d99-b45f-ea79d1c73453 admin admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:25.432378 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cc6af2-f6fe-4d99-b45f-ea79d1c73453 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:25.433506 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cc6af2-f6fe-4d99-b45f-ea79d1c73453 admin admin] PATH_INFO: `/v3/projects/6cc1a5eb5b294c0c964940dd83719ea0/users/123c3c729150460da6164952fc80a3ac/roles/c5346bc54abf445f9900edf94264c850` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:25.451891 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cc6af2-f6fe-4d99-b45f-ea79d1c73453 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6cc1a5eb5b294c0c964940dd83719ea0, user_id=123c3c729150460da6164952fc80a3ac, role_id=c5346bc54abf445f9900edf94264c850)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:25.454680 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cc6af2-f6fe-4d99-b45f-ea79d1c73453 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:25.561444 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72286aa5-90f0-4cf5-b433-722b73921b65 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:25.567507 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 908/1837] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 24 17:26:25 2022] PUT /identity/v3/projects/6cc1a5eb5b294c0c964940dd83719ea0/users/123c3c729150460da6164952fc80a3ac/roles/c5346bc54abf445f9900edf94264c850 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:26:25.573328 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72286aa5-90f0-4cf5-b433-722b73921b65 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:25.634722 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bab3fc-d694-4a6d-b019-76cd81e30602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:25.635454 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bab3fc-d694-4a6d-b019-76cd81e30602 admin admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:25.635932 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bab3fc-d694-4a6d-b019-76cd81e30602 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:25.636781 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bab3fc-d694-4a6d-b019-76cd81e30602 admin admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:25.637834 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bab3fc-d694-4a6d-b019-76cd81e30602 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:25.639505 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bab3fc-d694-4a6d-b019-76cd81e30602 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:25.651201 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 930/1838] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:26:25 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:26:25.655771 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cc6af2-f6fe-4d99-b45f-ea79d1c73453 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:25.667499 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cc6af2-f6fe-4d99-b45f-ea79d1c73453 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:25.674038 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca16dbb-c912-485d-b003-58bcc47f54de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:25.675096 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca16dbb-c912-485d-b003-58bcc47f54de admin admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:25.675096 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca16dbb-c912-485d-b003-58bcc47f54de admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:25.675371 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca16dbb-c912-485d-b003-58bcc47f54de admin admin] PATH_INFO: `/v3/projects/6cc1a5eb5b294c0c964940dd83719ea0/users/123c3c729150460da6164952fc80a3ac/roles/c5346bc54abf445f9900edf94264c850` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:25.677673 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca16dbb-c912-485d-b003-58bcc47f54de admin admin] RBAC: Authorizing `identity:create_grant(project_id=6cc1a5eb5b294c0c964940dd83719ea0, user_id=123c3c729150460da6164952fc80a3ac, role_id=c5346bc54abf445f9900edf94264c850)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:25.684693 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca16dbb-c912-485d-b003-58bcc47f54de admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:25.696049 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 909/1839] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 24 17:26:25 2022] PUT /identity/v3/projects/6cc1a5eb5b294c0c964940dd83719ea0/users/123c3c729150460da6164952fc80a3ac/roles/c5346bc54abf445f9900edf94264c850 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:26:25.701192 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:26:25.707892 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd98e2e-aa11-448c-9966-14338024af44 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:25.709673 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd98e2e-aa11-448c-9966-14338024af44 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:25.709673 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd98e2e-aa11-448c-9966-14338024af44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:25.736980 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-9cd98e2e-aa11-448c-9966-14338024af44 None None] MFA Rules not processed for user `123c3c729150460da6164952fc80a3ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:26:25.738238 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-9cd98e2e-aa11-448c-9966-14338024af44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:26:25.831680 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 931/1840] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:26:25 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:26:25.841369 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca16dbb-c912-485d-b003-58bcc47f54de admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:25.853214 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-6ca16dbb-c912-485d-b003-58bcc47f54de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:26:25.917689 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca16dbb-c912-485d-b003-58bcc47f54de admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:25.921248 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3302d75f-1938-4129-881a-81f313d12ae8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be06246d47584b529150ceb98a5cfd95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:25.921864 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3302d75f-1938-4129-881a-81f313d12ae8 service cinder] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:25.922145 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3302d75f-1938-4129-881a-81f313d12ae8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:25.922416 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3302d75f-1938-4129-881a-81f313d12ae8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:25.931861 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3302d75f-1938-4129-881a-81f313d12ae8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:25.933336 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3302d75f-1938-4129-881a-81f313d12ae8 service cinder] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:25.943087 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 910/1841] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:26:25 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:26:26.368772 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:26:26.381155 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3086f21b-a80e-4e44-ae3e-a2245f810d08 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:26.381597 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3086f21b-a80e-4e44-ae3e-a2245f810d08 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:26.381687 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3086f21b-a80e-4e44-ae3e-a2245f810d08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:26.411857 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-3086f21b-a80e-4e44-ae3e-a2245f810d08 None None] MFA Rules not processed for user `427a581c9c47411db485028e40830208`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:26:26.413266 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-3086f21b-a80e-4e44-ae3e-a2245f810d08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:26:26.490113 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 932/1842] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:26:26 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:26:26.494328 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3302d75f-1938-4129-881a-81f313d12ae8 service cinder] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:26.496335 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-3302d75f-1938-4129-881a-81f313d12ae8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:26:26.578058 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3302d75f-1938-4129-881a-81f313d12ae8 service cinder] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:26.585402 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e20e312-aac3-4f01-beb9-dd593b2c5923 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:26.587538 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e20e312-aac3-4f01-beb9-dd593b2c5923 admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:26.588046 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e20e312-aac3-4f01-beb9-dd593b2c5923 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:26.588523 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e20e312-aac3-4f01-beb9-dd593b2c5923 admin admin] PATH_INFO: `/v3/projects` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:26.589382 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e20e312-aac3-4f01-beb9-dd593b2c5923 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:26.591915 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e20e312-aac3-4f01-beb9-dd593b2c5923 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:26.716356 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:26:26.729547 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 911/1843] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 24 17:26:26 2022] POST /identity/v3/projects => generated 371 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:26:26.730853 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3086f21b-a80e-4e44-ae3e-a2245f810d08 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:26.743422 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3086f21b-a80e-4e44-ae3e-a2245f810d08 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:26.827868 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0889b5-1b04-42a8-bdbd-0e0a65965e56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:26.828645 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0889b5-1b04-42a8-bdbd-0e0a65965e56 admin admin] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:26.829013 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0889b5-1b04-42a8-bdbd-0e0a65965e56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:26.829373 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0889b5-1b04-42a8-bdbd-0e0a65965e56 admin admin] PATH_INFO: `/v3/users` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:26.830015 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0889b5-1b04-42a8-bdbd-0e0a65965e56 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:26.831987 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0889b5-1b04-42a8-bdbd-0e0a65965e56 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:26.955460 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 933/1844] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 24 17:26:26 2022] POST /identity/v3/users => generated 292 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:26:26.958775 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e20e312-aac3-4f01-beb9-dd593b2c5923 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:26.970752 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e20e312-aac3-4f01-beb9-dd593b2c5923 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:26.980092 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89df815-b742-4450-8e96-69425ac68808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:26.984011 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89df815-b742-4450-8e96-69425ac68808 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:26.984011 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89df815-b742-4450-8e96-69425ac68808 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:26.984011 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89df815-b742-4450-8e96-69425ac68808 admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:26.984011 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89df815-b742-4450-8e96-69425ac68808 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:26.985785 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89df815-b742-4450-8e96-69425ac68808 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:27.000340 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 912/1845] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:26:26 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:26:27.014546 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0889b5-1b04-42a8-bdbd-0e0a65965e56 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:27.039005 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0889b5-1b04-42a8-bdbd-0e0a65965e56 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:27.048681 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a375b590-9b3c-4027-9571-8ec1c07ff0d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:27.049365 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a375b590-9b3c-4027-9571-8ec1c07ff0d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:27.049643 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a375b590-9b3c-4027-9571-8ec1c07ff0d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:27.049916 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a375b590-9b3c-4027-9571-8ec1c07ff0d9 admin admin] PATH_INFO: `/v3/projects/3d3d9d27fbbd48549a012a01bc4392e2/users/8b1733d114e84704ad599efcc7b4c7be/roles/c5346bc54abf445f9900edf94264c850` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:27.066858 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a375b590-9b3c-4027-9571-8ec1c07ff0d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d3d9d27fbbd48549a012a01bc4392e2, user_id=8b1733d114e84704ad599efcc7b4c7be, role_id=c5346bc54abf445f9900edf94264c850)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:27.069586 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a375b590-9b3c-4027-9571-8ec1c07ff0d9 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:27.230277 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 934/1846] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 24 17:26:27 2022] PUT /identity/v3/projects/3d3d9d27fbbd48549a012a01bc4392e2/users/8b1733d114e84704ad599efcc7b4c7be/roles/c5346bc54abf445f9900edf94264c850 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:26:27.233806 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89df815-b742-4450-8e96-69425ac68808 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:27.257147 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89df815-b742-4450-8e96-69425ac68808 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:27.368890 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5826bd1-b19e-499a-9fbe-b3a448ed19ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:27.369891 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5826bd1-b19e-499a-9fbe-b3a448ed19ff admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:27.370245 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5826bd1-b19e-499a-9fbe-b3a448ed19ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:27.371184 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5826bd1-b19e-499a-9fbe-b3a448ed19ff admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:27.371925 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5826bd1-b19e-499a-9fbe-b3a448ed19ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:27.374017 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5826bd1-b19e-499a-9fbe-b3a448ed19ff admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:27.402735 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 913/1847] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:26:27 2022] GET /identity/v3/roles => generated 1420 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:26:27.404547 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a375b590-9b3c-4027-9571-8ec1c07ff0d9 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:27.417090 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a375b590-9b3c-4027-9571-8ec1c07ff0d9 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:27.427183 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c52587-5495-41fe-aa88-c2a3bc444a64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:27.427183 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c52587-5495-41fe-aa88-c2a3bc444a64 admin admin] REQUEST_METHOD: `PUT` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:27.427183 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c52587-5495-41fe-aa88-c2a3bc444a64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:27.427183 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c52587-5495-41fe-aa88-c2a3bc444a64 admin admin] PATH_INFO: `/v3/projects/3d3d9d27fbbd48549a012a01bc4392e2/users/8b1733d114e84704ad599efcc7b4c7be/roles/c5346bc54abf445f9900edf94264c850` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:27.428422 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c52587-5495-41fe-aa88-c2a3bc444a64 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d3d9d27fbbd48549a012a01bc4392e2, user_id=8b1733d114e84704ad599efcc7b4c7be, role_id=c5346bc54abf445f9900edf94264c850)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:27.432681 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c52587-5495-41fe-aa88-c2a3bc444a64 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:27.472845 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 935/1848] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 24 17:26:27 2022] PUT /identity/v3/projects/3d3d9d27fbbd48549a012a01bc4392e2/users/8b1733d114e84704ad599efcc7b4c7be/roles/c5346bc54abf445f9900edf94264c850 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:26:27.524739 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5826bd1-b19e-499a-9fbe-b3a448ed19ff admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:27.526619 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-c5826bd1-b19e-499a-9fbe-b3a448ed19ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:26:27.624555 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5826bd1-b19e-499a-9fbe-b3a448ed19ff admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:27.695005 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac450e2-d8e3-4afe-a85c-8f6393ec655c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be06246d47584b529150ceb98a5cfd95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:27.699246 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac450e2-d8e3-4afe-a85c-8f6393ec655c service cinder] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:27.699246 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac450e2-d8e3-4afe-a85c-8f6393ec655c service cinder] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:27.699246 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac450e2-d8e3-4afe-a85c-8f6393ec655c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:27.699696 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-6ac450e2-d8e3-4afe-a85c-8f6393ec655c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:26:27.792179 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac450e2-d8e3-4afe-a85c-8f6393ec655c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:27.799042 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac450e2-d8e3-4afe-a85c-8f6393ec655c service cinder] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:27.800206 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c52587-5495-41fe-aa88-c2a3bc444a64 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:27.812171 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 914/1849] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:26:27 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:26:27.900267 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c52587-5495-41fe-aa88-c2a3bc444a64 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:27.905425 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d2c7bc-2cac-4345-9982-f926c18869d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:27.906073 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d2c7bc-2cac-4345-9982-f926c18869d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:27.906362 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d2c7bc-2cac-4345-9982-f926c18869d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:27.906644 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d2c7bc-2cac-4345-9982-f926c18869d4 admin admin] PATH_INFO: `/v3/users/f37e50db294945c68d26d24da695e5f8` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:27.907880 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d2c7bc-2cac-4345-9982-f926c18869d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f37e50db294945c68d26d24da695e5f8)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:27.909804 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d2c7bc-2cac-4345-9982-f926c18869d4 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:28.155443 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-84d2c7bc-2cac-4345-9982-f926c18869d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f37e50db294945c68d26d24da695e5f8'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:28.172925 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-84d2c7bc-2cac-4345-9982-f926c18869d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f37e50db294945c68d26d24da695e5f8'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:28.237253 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:26:28.263548 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-84d2c7bc-2cac-4345-9982-f926c18869d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f37e50db294945c68d26d24da695e5f8'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:28.264127 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-84d2c7bc-2cac-4345-9982-f926c18869d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f37e50db294945c68d26d24da695e5f8'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:28.272509 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-84d2c7bc-2cac-4345-9982-f926c18869d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f37e50db294945c68d26d24da695e5f8'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:28.287107 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 936/1850] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 24 17:26:27 2022] DELETE /identity/v3/users/f37e50db294945c68d26d24da695e5f8 => generated 0 bytes in 484 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:26:28.288558 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3771b223-d8dd-403a-ad17-36b6b84af204 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:28.289508 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3771b223-d8dd-403a-ad17-36b6b84af204 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:28.289508 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3771b223-d8dd-403a-ad17-36b6b84af204 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:28.326955 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-3771b223-d8dd-403a-ad17-36b6b84af204 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:26:28.326955 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-3771b223-d8dd-403a-ad17-36b6b84af204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:26:28.391175 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 915/1851] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:26:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:26:28.394755 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:26:28.410703 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d2c7bc-2cac-4345-9982-f926c18869d4 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:28.412234 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-84d2c7bc-2cac-4345-9982-f926c18869d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:26:28.476382 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d2c7bc-2cac-4345-9982-f926c18869d4 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:28.480433 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dd96b0-75c6-49eb-a17a-339c998a99f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e9ed4d47aef4140bfb2d3bb798cc559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:28.481345 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dd96b0-75c6-49eb-a17a-339c998a99f2 service neutron] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:28.486508 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dd96b0-75c6-49eb-a17a-339c998a99f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:28.486508 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dd96b0-75c6-49eb-a17a-339c998a99f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:28.495861 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dd96b0-75c6-49eb-a17a-339c998a99f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:28.497296 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dd96b0-75c6-49eb-a17a-339c998a99f2 service neutron] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:28.510232 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 937/1852] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:26:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:26:29.036036 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3771b223-d8dd-403a-ad17-36b6b84af204 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:29.039276 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-3771b223-d8dd-403a-ad17-36b6b84af204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:26:29.060230 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3771b223-d8dd-403a-ad17-36b6b84af204 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:29.068111 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71f18b3-5ed9-4d73-b180-bf63cec6a724 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:29.070950 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71f18b3-5ed9-4d73-b180-bf63cec6a724 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:29.070950 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71f18b3-5ed9-4d73-b180-bf63cec6a724 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:29.072995 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71f18b3-5ed9-4d73-b180-bf63cec6a724 admin admin] PATH_INFO: `/v3/projects/95268d18617f41f186d2e95c3d1445c4` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:29.077042 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71f18b3-5ed9-4d73-b180-bf63cec6a724 admin admin] RBAC: Authorizing `identity:delete_project(project_id=95268d18617f41f186d2e95c3d1445c4)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:29.100255 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71f18b3-5ed9-4d73-b180-bf63cec6a724 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:29.105610 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3588edd-d21c-4f52-a53c-d7d6d3eba89a None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:29.106928 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3588edd-d21c-4f52-a53c-d7d6d3eba89a None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:29.109565 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3588edd-d21c-4f52-a53c-d7d6d3eba89a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:29.157459 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-d3588edd-d21c-4f52-a53c-d7d6d3eba89a None None] MFA Rules not processed for user `8b1733d114e84704ad599efcc7b4c7be`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:26:29.158762 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-d3588edd-d21c-4f52-a53c-d7d6d3eba89a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:26:29.239410 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-e71f18b3-5ed9-4d73-b180-bf63cec6a724 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95268d18617f41f186d2e95c3d1445c4', 'request_id': 'req-e71f18b3-5ed9-4d73-b180-bf63cec6a724'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:29.251585 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 938/1853] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:26:29 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:26:29.264343 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3588edd-d21c-4f52-a53c-d7d6d3eba89a None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:29.265767 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-d3588edd-d21c-4f52-a53c-d7d6d3eba89a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:26:29.320904 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3588edd-d21c-4f52-a53c-d7d6d3eba89a None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:29.334846 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-e71f18b3-5ed9-4d73-b180-bf63cec6a724 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95268d18617f41f186d2e95c3d1445c4', 'request_id': 'req-e71f18b3-5ed9-4d73-b180-bf63cec6a724'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:29.342575 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-e71f18b3-5ed9-4d73-b180-bf63cec6a724 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95268d18617f41f186d2e95c3d1445c4', 'request_id': 'req-e71f18b3-5ed9-4d73-b180-bf63cec6a724'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:29.355487 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-e71f18b3-5ed9-4d73-b180-bf63cec6a724 admin admin] The token cache is being invalidate because project 95268d18617f41f186d2e95c3d1445c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81732) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:26:29.355998 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-e71f18b3-5ed9-4d73-b180-bf63cec6a724 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:29.356669 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-e71f18b3-5ed9-4d73-b180-bf63cec6a724 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:29.359139 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 916/1854] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 24 17:26:29 2022] DELETE /identity/v3/projects/95268d18617f41f186d2e95c3d1445c4 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:26:29.366758 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71f18b3-5ed9-4d73-b180-bf63cec6a724 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:29.368405 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-e71f18b3-5ed9-4d73-b180-bf63cec6a724 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:26:29.383902 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde030e0-13ef-407d-b584-81e69a661f1f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be06246d47584b529150ceb98a5cfd95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:29.385369 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde030e0-13ef-407d-b584-81e69a661f1f service cinder] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:29.385654 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde030e0-13ef-407d-b584-81e69a661f1f service cinder] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:29.386598 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde030e0-13ef-407d-b584-81e69a661f1f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:29.388737 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-bde030e0-13ef-407d-b584-81e69a661f1f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:26:29.419032 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:26:29.451589 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71f18b3-5ed9-4d73-b180-bf63cec6a724 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:29.456498 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91beaea2-e294-41e6-bc9e-572ee1282601 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:29.457225 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91beaea2-e294-41e6-bc9e-572ee1282601 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:29.457603 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91beaea2-e294-41e6-bc9e-572ee1282601 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:29.457953 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91beaea2-e294-41e6-bc9e-572ee1282601 admin admin] PATH_INFO: `/v3/users/123c3c729150460da6164952fc80a3ac` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:29.459323 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91beaea2-e294-41e6-bc9e-572ee1282601 admin admin] RBAC: Authorizing `identity:delete_user(user_id=123c3c729150460da6164952fc80a3ac)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:29.461221 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde030e0-13ef-407d-b584-81e69a661f1f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:29.461697 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91beaea2-e294-41e6-bc9e-572ee1282601 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:29.462658 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde030e0-13ef-407d-b584-81e69a661f1f service cinder] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:29.472385 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 939/1855] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:26:29 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:26:29.637458 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:26:29.749922 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-91beaea2-e294-41e6-bc9e-572ee1282601 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '123c3c729150460da6164952fc80a3ac'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:30.944519 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-91beaea2-e294-41e6-bc9e-572ee1282601 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '123c3c729150460da6164952fc80a3ac'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:30.945647 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-91beaea2-e294-41e6-bc9e-572ee1282601 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '123c3c729150460da6164952fc80a3ac'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:30.955576 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-91beaea2-e294-41e6-bc9e-572ee1282601 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '123c3c729150460da6164952fc80a3ac'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:30.956313 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-91beaea2-e294-41e6-bc9e-572ee1282601 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '123c3c729150460da6164952fc80a3ac'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:30.972866 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 917/1856] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 24 17:26:29 2022] DELETE /identity/v3/users/123c3c729150460da6164952fc80a3ac => generated 0 bytes in 1608 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:26:31.647966 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde030e0-13ef-407d-b584-81e69a661f1f service cinder] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:31.671641 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde030e0-13ef-407d-b584-81e69a661f1f service cinder] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:31.749425 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640e0f7d-e12a-4479-a0e0-06d980118153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:31.750116 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640e0f7d-e12a-4479-a0e0-06d980118153 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:31.750395 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640e0f7d-e12a-4479-a0e0-06d980118153 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:31.750669 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640e0f7d-e12a-4479-a0e0-06d980118153 admin admin] PATH_INFO: `/v3/projects/6cc1a5eb5b294c0c964940dd83719ea0` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:31.751954 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640e0f7d-e12a-4479-a0e0-06d980118153 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6cc1a5eb5b294c0c964940dd83719ea0)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:31.753875 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640e0f7d-e12a-4479-a0e0-06d980118153 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:31.913399 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-640e0f7d-e12a-4479-a0e0-06d980118153 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6cc1a5eb5b294c0c964940dd83719ea0', 'request_id': 'req-640e0f7d-e12a-4479-a0e0-06d980118153'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:31.920260 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:26:31.983123 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-640e0f7d-e12a-4479-a0e0-06d980118153 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6cc1a5eb5b294c0c964940dd83719ea0', 'request_id': 'req-640e0f7d-e12a-4479-a0e0-06d980118153'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:31.990532 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-640e0f7d-e12a-4479-a0e0-06d980118153 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6cc1a5eb5b294c0c964940dd83719ea0', 'request_id': 'req-640e0f7d-e12a-4479-a0e0-06d980118153'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:32.002250 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-640e0f7d-e12a-4479-a0e0-06d980118153 admin admin] The token cache is being invalidate because project 6cc1a5eb5b294c0c964940dd83719ea0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81731) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:26:32.002539 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-640e0f7d-e12a-4479-a0e0-06d980118153 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:32.003239 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-640e0f7d-e12a-4479-a0e0-06d980118153 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:32.004825 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 940/1857] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 24 17:26:31 2022] DELETE /identity/v3/projects/6cc1a5eb5b294c0c964940dd83719ea0 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:26:32.010243 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d608266e-8bf2-4069-85a9-e7bc58e5b82a None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:32.011319 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d608266e-8bf2-4069-85a9-e7bc58e5b82a None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:32.011319 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d608266e-8bf2-4069-85a9-e7bc58e5b82a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:32.033206 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-d608266e-8bf2-4069-85a9-e7bc58e5b82a None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:26:32.034441 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-d608266e-8bf2-4069-85a9-e7bc58e5b82a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:26:32.100926 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 918/1858] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:26:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:26:32.105302 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61df2e37-627a-4c26-acf5-8f562a10dec5 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:32.105584 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61df2e37-627a-4c26-acf5-8f562a10dec5 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:32.105854 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61df2e37-627a-4c26-acf5-8f562a10dec5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:32.126728 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-61df2e37-627a-4c26-acf5-8f562a10dec5 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:26:32.128032 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-61df2e37-627a-4c26-acf5-8f562a10dec5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:26:32.137276 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 941/1859] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:26:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:26:32.142122 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d608266e-8bf2-4069-85a9-e7bc58e5b82a None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:32.156057 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:26:32.159200 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d608266e-8bf2-4069-85a9-e7bc58e5b82a None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:32.166887 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0051c8-b894-4431-b0b1-093118d22542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:32.167846 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0051c8-b894-4431-b0b1-093118d22542 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:32.168276 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0051c8-b894-4431-b0b1-093118d22542 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:32.168748 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0051c8-b894-4431-b0b1-093118d22542 admin admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:32.169430 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0051c8-b894-4431-b0b1-093118d22542 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:26:32.170289 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0051c8-b894-4431-b0b1-093118d22542 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:32.172047 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0051c8-b894-4431-b0b1-093118d22542 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:32.189480 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 919/1860] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 24 17:26:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:26:32.194550 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61df2e37-627a-4c26-acf5-8f562a10dec5 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:32.205169 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61df2e37-627a-4c26-acf5-8f562a10dec5 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:32.211913 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58785ca-a4d9-49de-8075-6ec5ebc3849d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:32.212597 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58785ca-a4d9-49de-8075-6ec5ebc3849d admin admin] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:32.212853 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58785ca-a4d9-49de-8075-6ec5ebc3849d admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:32.213970 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58785ca-a4d9-49de-8075-6ec5ebc3849d admin admin] PATH_INFO: `/v3/projects` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:32.214721 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58785ca-a4d9-49de-8075-6ec5ebc3849d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:32.216740 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58785ca-a4d9-49de-8075-6ec5ebc3849d admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:32.332074 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 942/1861] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 24 17:26:32 2022] POST /identity/v3/projects => generated 369 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:26:32.336232 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0051c8-b894-4431-b0b1-093118d22542 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:32.346502 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0051c8-b894-4431-b0b1-093118d22542 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:32.406175 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d8a051-6656-4c28-b185-a9fe8f323df6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:32.406946 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d8a051-6656-4c28-b185-a9fe8f323df6 admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:32.411571 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d8a051-6656-4c28-b185-a9fe8f323df6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:32.413558 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d8a051-6656-4c28-b185-a9fe8f323df6 admin admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:32.413558 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d8a051-6656-4c28-b185-a9fe8f323df6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:32.414675 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d8a051-6656-4c28-b185-a9fe8f323df6 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:32.499839 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 920/1862] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 24 17:26:32 2022] POST /identity/v3/users => generated 285 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:26:32.503495 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58785ca-a4d9-49de-8075-6ec5ebc3849d admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:32.514546 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58785ca-a4d9-49de-8075-6ec5ebc3849d admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:32.519594 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69e04f4-1299-4da7-a6ba-809cbd5f367c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:32.520193 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69e04f4-1299-4da7-a6ba-809cbd5f367c admin admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:32.520699 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69e04f4-1299-4da7-a6ba-809cbd5f367c admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:32.520858 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69e04f4-1299-4da7-a6ba-809cbd5f367c admin admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:32.521510 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69e04f4-1299-4da7-a6ba-809cbd5f367c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:32.523115 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69e04f4-1299-4da7-a6ba-809cbd5f367c admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:32.544231 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 943/1863] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:26:32 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:26:32.546653 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d8a051-6656-4c28-b185-a9fe8f323df6 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:32.557588 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d8a051-6656-4c28-b185-a9fe8f323df6 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:32.566779 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17f9172-2af3-434f-8cea-a2526f09b251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:32.568408 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17f9172-2af3-434f-8cea-a2526f09b251 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:32.568408 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17f9172-2af3-434f-8cea-a2526f09b251 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:32.568408 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17f9172-2af3-434f-8cea-a2526f09b251 admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:32.568814 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17f9172-2af3-434f-8cea-a2526f09b251 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:32.570693 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17f9172-2af3-434f-8cea-a2526f09b251 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:32.593157 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 921/1864] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:26:32 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:26:32.594626 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69e04f4-1299-4da7-a6ba-809cbd5f367c admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:32.616417 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69e04f4-1299-4da7-a6ba-809cbd5f367c admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:32.624889 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b375ccf-aef2-4f2c-9c05-d90f5bfd6e45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:32.625943 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b375ccf-aef2-4f2c-9c05-d90f5bfd6e45 admin admin] REQUEST_METHOD: `PUT` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:32.626404 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b375ccf-aef2-4f2c-9c05-d90f5bfd6e45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:32.626991 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b375ccf-aef2-4f2c-9c05-d90f5bfd6e45 admin admin] PATH_INFO: `/v3/projects/e030c844d6544b4cbb9a9f8339d2c72c/users/a1a0b876df274ea08e8c95c9b7f5a959/roles/60dc7e16aee746d5a3e8aa27d7d8525c` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:32.645396 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b375ccf-aef2-4f2c-9c05-d90f5bfd6e45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e030c844d6544b4cbb9a9f8339d2c72c, user_id=a1a0b876df274ea08e8c95c9b7f5a959, role_id=60dc7e16aee746d5a3e8aa27d7d8525c)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:32.648235 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b375ccf-aef2-4f2c-9c05-d90f5bfd6e45 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:32.746715 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 944/1865] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 24 17:26:32 2022] PUT /identity/v3/projects/e030c844d6544b4cbb9a9f8339d2c72c/users/a1a0b876df274ea08e8c95c9b7f5a959/roles/60dc7e16aee746d5a3e8aa27d7d8525c => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:26:32.757238 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51473a29-1017-4bc7-a088-a9e9761aebc5 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:32.757526 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51473a29-1017-4bc7-a088-a9e9761aebc5 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:32.757799 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51473a29-1017-4bc7-a088-a9e9761aebc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:32.793474 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-51473a29-1017-4bc7-a088-a9e9761aebc5 None None] MFA Rules not processed for user `a1a0b876df274ea08e8c95c9b7f5a959`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:26:32.795618 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-51473a29-1017-4bc7-a088-a9e9761aebc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:26:32.884578 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 922/1866] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:26:32 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:26:32.897726 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b375ccf-aef2-4f2c-9c05-d90f5bfd6e45 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:32.899637 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-7b375ccf-aef2-4f2c-9c05-d90f5bfd6e45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:26:32.997114 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b375ccf-aef2-4f2c-9c05-d90f5bfd6e45 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:33.001059 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f96a961-4744-4a1c-a82f-e5116f6e0dc6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3553e522363046f6828fd61b2a0642ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:33.001725 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f96a961-4744-4a1c-a82f-e5116f6e0dc6 service glance] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:33.002003 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f96a961-4744-4a1c-a82f-e5116f6e0dc6 service glance] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:33.002275 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f96a961-4744-4a1c-a82f-e5116f6e0dc6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:33.028502 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f96a961-4744-4a1c-a82f-e5116f6e0dc6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:33.034041 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f96a961-4744-4a1c-a82f-e5116f6e0dc6 service glance] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:33.056566 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 945/1867] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:26:32 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:26:33.642975 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51473a29-1017-4bc7-a088-a9e9761aebc5 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:33.644742 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-51473a29-1017-4bc7-a088-a9e9761aebc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:26:33.711988 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51473a29-1017-4bc7-a088-a9e9761aebc5 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:33.715743 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a7c577-9d00-48f6-baae-84287ac8b2ff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be06246d47584b529150ceb98a5cfd95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:33.716366 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a7c577-9d00-48f6-baae-84287ac8b2ff service cinder] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:33.716645 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a7c577-9d00-48f6-baae-84287ac8b2ff service cinder] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:33.716909 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a7c577-9d00-48f6-baae-84287ac8b2ff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:33.718809 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-91a7c577-9d00-48f6-baae-84287ac8b2ff service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:26:33.799855 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a7c577-9d00-48f6-baae-84287ac8b2ff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:33.801459 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a7c577-9d00-48f6-baae-84287ac8b2ff service cinder] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:33.811741 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 923/1868] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:26:33 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:26:34.608593 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:26:34.718092 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9f7799-2569-4fcd-9754-34213976b876 None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:34.718688 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9f7799-2569-4fcd-9754-34213976b876 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:34.718688 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9f7799-2569-4fcd-9754-34213976b876 None None] PATH_INFO: `/` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:34.723281 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 946/1869] 10.222.0.18 () {54 vars in 921 bytes} [Wed Aug 24 17:26:34 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:26:34.745438 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64480d1b-090a-456b-a9da-dd7a7a603e9f None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:34.745915 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64480d1b-090a-456b-a9da-dd7a7a603e9f None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:34.746032 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64480d1b-090a-456b-a9da-dd7a7a603e9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:34.780442 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-64480d1b-090a-456b-a9da-dd7a7a603e9f None None] MFA Rules not processed for user `557aca9255cb4bbb8af52e796c5c5177`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:26:34.783903 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-64480d1b-090a-456b-a9da-dd7a7a603e9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:26:34.923994 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 924/1870] 10.222.0.18 () {58 vars in 1018 bytes} [Wed Aug 24 17:26:34 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:26:34.946556 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:26:35.107465 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9f7799-2569-4fcd-9754-34213976b876 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:35.112645 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-ea9f7799-2569-4fcd-9754-34213976b876 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:26:35.144571 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9f7799-2569-4fcd-9754-34213976b876 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:35.150642 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fe7dfb-1d7c-4e7a-b4d3-f7eb6f9f2c47 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557aca9255cb4bbb8af52e796c5c5177', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:35.151310 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fe7dfb-1d7c-4e7a-b4d3-f7eb6f9f2c47 service nova] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:35.152159 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fe7dfb-1d7c-4e7a-b4d3-f7eb6f9f2c47 service nova] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:35.152439 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fe7dfb-1d7c-4e7a-b4d3-f7eb6f9f2c47 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:35.169634 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fe7dfb-1d7c-4e7a-b4d3-f7eb6f9f2c47 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:35.173164 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fe7dfb-1d7c-4e7a-b4d3-f7eb6f9f2c47 service nova] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:35.190978 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 947/1871] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:26:35 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:26:35.496645 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64480d1b-090a-456b-a9da-dd7a7a603e9f None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:35.499344 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-64480d1b-090a-456b-a9da-dd7a7a603e9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:26:35.589708 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64480d1b-090a-456b-a9da-dd7a7a603e9f None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:35.596995 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ccdf1f-6052-4bbd-9098-fe293f8a9d84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:35.597703 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ccdf1f-6052-4bbd-9098-fe293f8a9d84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:35.597988 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ccdf1f-6052-4bbd-9098-fe293f8a9d84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:35.599252 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ccdf1f-6052-4bbd-9098-fe293f8a9d84 admin admin] PATH_INFO: `/v3/users/b5c65900d2e64e96b699024e86474d72` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:35.600431 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ccdf1f-6052-4bbd-9098-fe293f8a9d84 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5c65900d2e64e96b699024e86474d72)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:35.602467 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ccdf1f-6052-4bbd-9098-fe293f8a9d84 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:36.005565 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-22ccdf1f-6052-4bbd-9098-fe293f8a9d84 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5c65900d2e64e96b699024e86474d72'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:36.134540 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-22ccdf1f-6052-4bbd-9098-fe293f8a9d84 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5c65900d2e64e96b699024e86474d72'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:36.135513 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-22ccdf1f-6052-4bbd-9098-fe293f8a9d84 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5c65900d2e64e96b699024e86474d72'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:36.155089 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-22ccdf1f-6052-4bbd-9098-fe293f8a9d84 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5c65900d2e64e96b699024e86474d72'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:36.156416 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-22ccdf1f-6052-4bbd-9098-fe293f8a9d84 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5c65900d2e64e96b699024e86474d72'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:36.163244 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:26:36.176372 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 925/1872] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 24 17:26:35 2022] DELETE /identity/v3/users/b5c65900d2e64e96b699024e86474d72 => generated 0 bytes in 681 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:26:36.180919 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a00e2f7-6f75-4d38-bb8f-cafbacf4e314 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:36.181205 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a00e2f7-6f75-4d38-bb8f-cafbacf4e314 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:36.181742 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a00e2f7-6f75-4d38-bb8f-cafbacf4e314 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:36.204820 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-4a00e2f7-6f75-4d38-bb8f-cafbacf4e314 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:26:36.206094 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-4a00e2f7-6f75-4d38-bb8f-cafbacf4e314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:26:36.293484 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 948/1873] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:26:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:26:36.304211 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ccdf1f-6052-4bbd-9098-fe293f8a9d84 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:36.305614 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-22ccdf1f-6052-4bbd-9098-fe293f8a9d84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:26:36.376869 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ccdf1f-6052-4bbd-9098-fe293f8a9d84 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:36.381728 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f160f6c-1092-4f49-9945-fa877a2fbdcc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e9ed4d47aef4140bfb2d3bb798cc559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:36.382396 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f160f6c-1092-4f49-9945-fa877a2fbdcc service neutron] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:36.382699 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f160f6c-1092-4f49-9945-fa877a2fbdcc service neutron] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:36.383061 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f160f6c-1092-4f49-9945-fa877a2fbdcc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:36.394989 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f160f6c-1092-4f49-9945-fa877a2fbdcc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:36.396756 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f160f6c-1092-4f49-9945-fa877a2fbdcc service neutron] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:36.425958 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 926/1874] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:26:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:26:37.085058 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a00e2f7-6f75-4d38-bb8f-cafbacf4e314 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:37.086528 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-4a00e2f7-6f75-4d38-bb8f-cafbacf4e314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:26:37.107100 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a00e2f7-6f75-4d38-bb8f-cafbacf4e314 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:37.112986 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cb31a0-3c0f-4b43-b58f-159e0572a642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:37.113557 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cb31a0-3c0f-4b43-b58f-159e0572a642 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:37.113900 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cb31a0-3c0f-4b43-b58f-159e0572a642 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:37.114235 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cb31a0-3c0f-4b43-b58f-159e0572a642 admin admin] PATH_INFO: `/v3/projects/97fcab3e3f6d45daa970b89b51b7d109` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:37.115588 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cb31a0-3c0f-4b43-b58f-159e0572a642 admin admin] RBAC: Authorizing `identity:delete_project(project_id=97fcab3e3f6d45daa970b89b51b7d109)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:37.118273 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cb31a0-3c0f-4b43-b58f-159e0572a642 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:37.278166 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-70cb31a0-3c0f-4b43-b58f-159e0572a642 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97fcab3e3f6d45daa970b89b51b7d109', 'request_id': 'req-70cb31a0-3c0f-4b43-b58f-159e0572a642'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:37.346132 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-70cb31a0-3c0f-4b43-b58f-159e0572a642 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97fcab3e3f6d45daa970b89b51b7d109', 'request_id': 'req-70cb31a0-3c0f-4b43-b58f-159e0572a642'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:37.353134 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-70cb31a0-3c0f-4b43-b58f-159e0572a642 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97fcab3e3f6d45daa970b89b51b7d109', 'request_id': 'req-70cb31a0-3c0f-4b43-b58f-159e0572a642'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:37.366301 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-70cb31a0-3c0f-4b43-b58f-159e0572a642 admin admin] The token cache is being invalidate because project 97fcab3e3f6d45daa970b89b51b7d109 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81731) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:26:37.366701 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-70cb31a0-3c0f-4b43-b58f-159e0572a642 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:37.378224 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-70cb31a0-3c0f-4b43-b58f-159e0572a642 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:37.382993 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 949/1875] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 24 17:26:37 2022] DELETE /identity/v3/projects/97fcab3e3f6d45daa970b89b51b7d109 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:26:37.384366 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f160f6c-1092-4f49-9945-fa877a2fbdcc service neutron] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:37.397991 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f160f6c-1092-4f49-9945-fa877a2fbdcc service neutron] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:37.506242 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118ffb9b-7fcc-4972-8a13-3ce67ad86c8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:37.507307 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118ffb9b-7fcc-4972-8a13-3ce67ad86c8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:37.507741 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118ffb9b-7fcc-4972-8a13-3ce67ad86c8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:37.508165 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118ffb9b-7fcc-4972-8a13-3ce67ad86c8e admin admin] PATH_INFO: `/v3/users/d7cf12946e364d318ef67f5276e3ff96` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:37.510508 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118ffb9b-7fcc-4972-8a13-3ce67ad86c8e admin admin] RBAC: Authorizing `identity:delete_user(user_id=d7cf12946e364d318ef67f5276e3ff96)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:37.513234 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118ffb9b-7fcc-4972-8a13-3ce67ad86c8e admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:37.841882 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-118ffb9b-7fcc-4972-8a13-3ce67ad86c8e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd7cf12946e364d318ef67f5276e3ff96'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:38.025919 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-118ffb9b-7fcc-4972-8a13-3ce67ad86c8e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd7cf12946e364d318ef67f5276e3ff96'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:38.027070 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-118ffb9b-7fcc-4972-8a13-3ce67ad86c8e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd7cf12946e364d318ef67f5276e3ff96'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:38.035581 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-118ffb9b-7fcc-4972-8a13-3ce67ad86c8e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd7cf12946e364d318ef67f5276e3ff96'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:38.043080 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-118ffb9b-7fcc-4972-8a13-3ce67ad86c8e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd7cf12946e364d318ef67f5276e3ff96'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:38.057678 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 927/1876] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 24 17:26:37 2022] DELETE /identity/v3/users/d7cf12946e364d318ef67f5276e3ff96 => generated 0 bytes in 674 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:26:38.097438 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:26:38.570958 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cb31a0-3c0f-4b43-b58f-159e0572a642 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:38.574575 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-70cb31a0-3c0f-4b43-b58f-159e0572a642 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:26:38.596766 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cb31a0-3c0f-4b43-b58f-159e0572a642 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:38.727662 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7324b71-957e-492c-acce-df1153cac342 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:38.730708 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7324b71-957e-492c-acce-df1153cac342 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:38.732853 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7324b71-957e-492c-acce-df1153cac342 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:38.733179 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7324b71-957e-492c-acce-df1153cac342 admin admin] PATH_INFO: `/v3/projects/059cd28fa7d74054bc807c2b7974073a` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:38.736529 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7324b71-957e-492c-acce-df1153cac342 admin admin] RBAC: Authorizing `identity:delete_project(project_id=059cd28fa7d74054bc807c2b7974073a)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:38.745129 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7324b71-957e-492c-acce-df1153cac342 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:38.928700 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-d7324b71-957e-492c-acce-df1153cac342 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '059cd28fa7d74054bc807c2b7974073a', 'request_id': 'req-d7324b71-957e-492c-acce-df1153cac342'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:38.960897 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:26:39.018244 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-d7324b71-957e-492c-acce-df1153cac342 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '059cd28fa7d74054bc807c2b7974073a', 'request_id': 'req-d7324b71-957e-492c-acce-df1153cac342'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:39.028758 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-d7324b71-957e-492c-acce-df1153cac342 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '059cd28fa7d74054bc807c2b7974073a', 'request_id': 'req-d7324b71-957e-492c-acce-df1153cac342'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:39.045839 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-d7324b71-957e-492c-acce-df1153cac342 admin admin] The token cache is being invalidate because project 059cd28fa7d74054bc807c2b7974073a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81731) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:26:39.046229 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-d7324b71-957e-492c-acce-df1153cac342 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:39.046952 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-d7324b71-957e-492c-acce-df1153cac342 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:39.048815 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 950/1877] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 24 17:26:38 2022] DELETE /identity/v3/projects/059cd28fa7d74054bc807c2b7974073a => generated 0 bytes in 478 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:26:39.055241 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e12ebdd-5b44-42f2-b490-27f23a85a5e1 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:39.056363 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e12ebdd-5b44-42f2-b490-27f23a85a5e1 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:39.056874 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e12ebdd-5b44-42f2-b490-27f23a85a5e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:39.097658 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-9e12ebdd-5b44-42f2-b490-27f23a85a5e1 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:26:39.099088 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-9e12ebdd-5b44-42f2-b490-27f23a85a5e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:26:39.195226 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 928/1878] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:26:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:26:39.203508 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ef5c5b-1241-4d8f-b93b-3acdff3332a6 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:39.203791 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ef5c5b-1241-4d8f-b93b-3acdff3332a6 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:39.204153 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ef5c5b-1241-4d8f-b93b-3acdff3332a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:39.253540 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-23ef5c5b-1241-4d8f-b93b-3acdff3332a6 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:26:39.257303 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-23ef5c5b-1241-4d8f-b93b-3acdff3332a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:26:39.272196 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 951/1879] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:26:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:26:39.278305 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e12ebdd-5b44-42f2-b490-27f23a85a5e1 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:39.296999 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e12ebdd-5b44-42f2-b490-27f23a85a5e1 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:39.304388 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fb8f99-42e7-4fe0-b1aa-1e61a6eb2920 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:39.305058 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fb8f99-42e7-4fe0-b1aa-1e61a6eb2920 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:39.305339 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fb8f99-42e7-4fe0-b1aa-1e61a6eb2920 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:39.305609 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fb8f99-42e7-4fe0-b1aa-1e61a6eb2920 admin admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:39.306326 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fb8f99-42e7-4fe0-b1aa-1e61a6eb2920 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:26:39.306635 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fb8f99-42e7-4fe0-b1aa-1e61a6eb2920 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:39.308485 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fb8f99-42e7-4fe0-b1aa-1e61a6eb2920 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:39.327893 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 929/1880] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 24 17:26:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:26:39.334546 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ef5c5b-1241-4d8f-b93b-3acdff3332a6 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:39.352832 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ef5c5b-1241-4d8f-b93b-3acdff3332a6 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:39.358214 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74864a7-484a-429d-9f51-3723ce13b06f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:39.358844 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74864a7-484a-429d-9f51-3723ce13b06f admin admin] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:39.359224 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74864a7-484a-429d-9f51-3723ce13b06f admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:39.365438 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74864a7-484a-429d-9f51-3723ce13b06f admin admin] PATH_INFO: `/v3/projects` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:39.365438 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74864a7-484a-429d-9f51-3723ce13b06f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:39.365438 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74864a7-484a-429d-9f51-3723ce13b06f admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:39.572211 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 952/1881] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 24 17:26:39 2022] POST /identity/v3/projects => generated 375 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:26:39.576564 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fb8f99-42e7-4fe0-b1aa-1e61a6eb2920 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:39.591697 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fb8f99-42e7-4fe0-b1aa-1e61a6eb2920 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:39.683965 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:26:39.692398 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8fef14-327c-4bb6-bf76-22e6ec39cb9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:39.693761 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8fef14-327c-4bb6-bf76-22e6ec39cb9d admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:39.694046 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8fef14-327c-4bb6-bf76-22e6ec39cb9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:39.694320 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8fef14-327c-4bb6-bf76-22e6ec39cb9d admin admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:39.695148 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8fef14-327c-4bb6-bf76-22e6ec39cb9d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:39.697405 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8fef14-327c-4bb6-bf76-22e6ec39cb9d admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:39.800117 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 930/1882] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 24 17:26:39 2022] POST /identity/v3/users => generated 288 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:26:39.804746 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74864a7-484a-429d-9f51-3723ce13b06f admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:39.840134 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74864a7-484a-429d-9f51-3723ce13b06f admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:39.868774 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d019b0d2-c1f1-4152-89b2-040c9bfc64d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:39.869638 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d019b0d2-c1f1-4152-89b2-040c9bfc64d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:39.869927 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d019b0d2-c1f1-4152-89b2-040c9bfc64d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:39.873776 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d019b0d2-c1f1-4152-89b2-040c9bfc64d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:39.873776 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d019b0d2-c1f1-4152-89b2-040c9bfc64d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:39.878658 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d019b0d2-c1f1-4152-89b2-040c9bfc64d4 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:39.912486 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 953/1883] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:26:39 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:26:39.918568 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8fef14-327c-4bb6-bf76-22e6ec39cb9d admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:39.930353 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8fef14-327c-4bb6-bf76-22e6ec39cb9d admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:39.954815 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f9f49a-856a-443a-8a32-7f8f3bfa185a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:39.955526 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f9f49a-856a-443a-8a32-7f8f3bfa185a admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:39.955820 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f9f49a-856a-443a-8a32-7f8f3bfa185a admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:39.956080 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f9f49a-856a-443a-8a32-7f8f3bfa185a admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:39.956760 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f9f49a-856a-443a-8a32-7f8f3bfa185a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:39.961149 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f9f49a-856a-443a-8a32-7f8f3bfa185a admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:39.979007 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 931/1884] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:26:39 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:26:39.982626 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d019b0d2-c1f1-4152-89b2-040c9bfc64d4 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:39.994605 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d019b0d2-c1f1-4152-89b2-040c9bfc64d4 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:40.001299 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8c58ab-d605-405c-ab20-9901558e6126 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:40.002160 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8c58ab-d605-405c-ab20-9901558e6126 admin admin] REQUEST_METHOD: `PUT` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:40.002561 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8c58ab-d605-405c-ab20-9901558e6126 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:40.003377 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8c58ab-d605-405c-ab20-9901558e6126 admin admin] PATH_INFO: `/v3/projects/5c0c4ff4b452459980ef8067737854bb/users/5227b74d457640a4b4db8346ae48e388/roles/60dc7e16aee746d5a3e8aa27d7d8525c` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:40.024700 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8c58ab-d605-405c-ab20-9901558e6126 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c0c4ff4b452459980ef8067737854bb, user_id=5227b74d457640a4b4db8346ae48e388, role_id=60dc7e16aee746d5a3e8aa27d7d8525c)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:40.027767 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8c58ab-d605-405c-ab20-9901558e6126 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:40.153437 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 954/1885] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 24 17:26:39 2022] PUT /identity/v3/projects/5c0c4ff4b452459980ef8067737854bb/users/5227b74d457640a4b4db8346ae48e388/roles/60dc7e16aee746d5a3e8aa27d7d8525c => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:26:40.166156 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74d977b-dcce-4a8f-a22d-5ee79818b8ff None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:40.169633 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74d977b-dcce-4a8f-a22d-5ee79818b8ff None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:40.170044 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74d977b-dcce-4a8f-a22d-5ee79818b8ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:40.255330 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-d74d977b-dcce-4a8f-a22d-5ee79818b8ff None None] MFA Rules not processed for user `5227b74d457640a4b4db8346ae48e388`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:26:40.263795 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-d74d977b-dcce-4a8f-a22d-5ee79818b8ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:26:40.439586 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 932/1886] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:26:40 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:26:40.442542 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8c58ab-d605-405c-ab20-9901558e6126 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:40.613582 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8c58ab-d605-405c-ab20-9901558e6126 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:40.619904 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddca21b-91f2-48e5-bb85-281fa830d9ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:40.622198 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddca21b-91f2-48e5-bb85-281fa830d9ff admin admin] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:40.622555 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddca21b-91f2-48e5-bb85-281fa830d9ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:40.623039 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddca21b-91f2-48e5-bb85-281fa830d9ff admin admin] PATH_INFO: `/v3/projects` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:40.626201 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddca21b-91f2-48e5-bb85-281fa830d9ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:40.629902 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddca21b-91f2-48e5-bb85-281fa830d9ff admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:40.742971 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 955/1887] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 24 17:26:40 2022] POST /identity/v3/projects => generated 375 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:26:40.744472 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74d977b-dcce-4a8f-a22d-5ee79818b8ff None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:40.761233 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74d977b-dcce-4a8f-a22d-5ee79818b8ff None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:40.891755 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0719a89-e6a1-4087-972e-a33faf177850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:40.892841 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0719a89-e6a1-4087-972e-a33faf177850 admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:40.893281 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0719a89-e6a1-4087-972e-a33faf177850 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:40.893719 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0719a89-e6a1-4087-972e-a33faf177850 admin admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:40.894527 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0719a89-e6a1-4087-972e-a33faf177850 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:40.897004 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0719a89-e6a1-4087-972e-a33faf177850 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:41.118412 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddca21b-91f2-48e5-bb85-281fa830d9ff admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:41.120287 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 933/1888] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 24 17:26:40 2022] POST /identity/v3/users => generated 294 bytes in 373 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:26:41.135936 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddca21b-91f2-48e5-bb85-281fa830d9ff admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:41.146101 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3decfa-2004-4228-9396-96b1ef255a17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:41.146859 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3decfa-2004-4228-9396-96b1ef255a17 admin admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:41.148035 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3decfa-2004-4228-9396-96b1ef255a17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:41.148035 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3decfa-2004-4228-9396-96b1ef255a17 admin admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:41.148838 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3decfa-2004-4228-9396-96b1ef255a17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:41.151076 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3decfa-2004-4228-9396-96b1ef255a17 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:41.165744 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 956/1889] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:26:41 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:26:41.171208 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0719a89-e6a1-4087-972e-a33faf177850 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:41.184159 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0719a89-e6a1-4087-972e-a33faf177850 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:41.189966 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0197545e-ab9e-4390-9a57-f17742f292db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:41.190983 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0197545e-ab9e-4390-9a57-f17742f292db admin admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:41.191422 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0197545e-ab9e-4390-9a57-f17742f292db admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:41.191845 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0197545e-ab9e-4390-9a57-f17742f292db admin admin] PATH_INFO: `/v3/projects/9d3d20b42a494a2185bed9102ecef015/users/e0f60d59ab7641c89397b055c2b5f86b/roles/c5346bc54abf445f9900edf94264c850` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:41.216484 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0197545e-ab9e-4390-9a57-f17742f292db admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d3d20b42a494a2185bed9102ecef015, user_id=e0f60d59ab7641c89397b055c2b5f86b, role_id=c5346bc54abf445f9900edf94264c850)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:41.221684 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0197545e-ab9e-4390-9a57-f17742f292db admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:41.303915 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 934/1890] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 24 17:26:41 2022] PUT /identity/v3/projects/9d3d20b42a494a2185bed9102ecef015/users/e0f60d59ab7641c89397b055c2b5f86b/roles/c5346bc54abf445f9900edf94264c850 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:26:41.308759 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3decfa-2004-4228-9396-96b1ef255a17 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:41.324878 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3decfa-2004-4228-9396-96b1ef255a17 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:41.459184 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43489edb-448e-4f43-b586-c93c9cdaa1c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:41.460300 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43489edb-448e-4f43-b586-c93c9cdaa1c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:41.460613 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43489edb-448e-4f43-b586-c93c9cdaa1c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:41.461610 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43489edb-448e-4f43-b586-c93c9cdaa1c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:41.462319 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43489edb-448e-4f43-b586-c93c9cdaa1c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:41.466324 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43489edb-448e-4f43-b586-c93c9cdaa1c6 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:41.489715 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 957/1891] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:26:41 2022] GET /identity/v3/roles => generated 1420 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:26:41.499413 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0197545e-ab9e-4390-9a57-f17742f292db admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:41.517944 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0197545e-ab9e-4390-9a57-f17742f292db admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:41.539934 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7e1965-71e4-42c6-87ff-c033df30cc33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:41.558186 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7e1965-71e4-42c6-87ff-c033df30cc33 admin admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:41.558776 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7e1965-71e4-42c6-87ff-c033df30cc33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:41.559586 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7e1965-71e4-42c6-87ff-c033df30cc33 admin admin] PATH_INFO: `/v3/projects/9d3d20b42a494a2185bed9102ecef015/users/e0f60d59ab7641c89397b055c2b5f86b/roles/c5346bc54abf445f9900edf94264c850` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:41.570369 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7e1965-71e4-42c6-87ff-c033df30cc33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d3d20b42a494a2185bed9102ecef015, user_id=e0f60d59ab7641c89397b055c2b5f86b, role_id=c5346bc54abf445f9900edf94264c850)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:41.581029 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7e1965-71e4-42c6-87ff-c033df30cc33 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:41.637038 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 935/1892] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 24 17:26:41 2022] PUT /identity/v3/projects/9d3d20b42a494a2185bed9102ecef015/users/e0f60d59ab7641c89397b055c2b5f86b/roles/c5346bc54abf445f9900edf94264c850 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:26:41.645937 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957936f8-bf45-4987-9f04-399f9f5c3af8 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:41.647783 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957936f8-bf45-4987-9f04-399f9f5c3af8 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:41.648513 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957936f8-bf45-4987-9f04-399f9f5c3af8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:41.702794 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:26:41.705769 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-957936f8-bf45-4987-9f04-399f9f5c3af8 None None] MFA Rules not processed for user `e0f60d59ab7641c89397b055c2b5f86b`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:26:41.715402 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-957936f8-bf45-4987-9f04-399f9f5c3af8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:26:41.859312 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 958/1893] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:26:41 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:26:41.874951 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7e1965-71e4-42c6-87ff-c033df30cc33 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:41.877530 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-0e7e1965-71e4-42c6-87ff-c033df30cc33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:26:41.996781 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7e1965-71e4-42c6-87ff-c033df30cc33 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:42.001626 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bea785-04aa-4871-98b4-e7a33d1ad2e3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be06246d47584b529150ceb98a5cfd95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:42.002346 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bea785-04aa-4871-98b4-e7a33d1ad2e3 service cinder] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:42.002629 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bea785-04aa-4871-98b4-e7a33d1ad2e3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:42.002895 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bea785-04aa-4871-98b4-e7a33d1ad2e3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:42.084262 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:26:42.111808 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bea785-04aa-4871-98b4-e7a33d1ad2e3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:42.113688 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bea785-04aa-4871-98b4-e7a33d1ad2e3 service cinder] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:42.128896 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 936/1894] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:26:41 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:26:44.922785 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:26:48.937699 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957936f8-bf45-4987-9f04-399f9f5c3af8 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:49.068652 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957936f8-bf45-4987-9f04-399f9f5c3af8 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:49.078340 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ae4168-8d0b-4cd2-aaf3-98ed335cd089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:49.079485 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ae4168-8d0b-4cd2-aaf3-98ed335cd089 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:49.079987 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ae4168-8d0b-4cd2-aaf3-98ed335cd089 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:49.080492 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ae4168-8d0b-4cd2-aaf3-98ed335cd089 admin admin] PATH_INFO: `/v3/users/5227b74d457640a4b4db8346ae48e388` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:49.082671 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ae4168-8d0b-4cd2-aaf3-98ed335cd089 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5227b74d457640a4b4db8346ae48e388)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:49.086321 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ae4168-8d0b-4cd2-aaf3-98ed335cd089 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:49.297446 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:26:49.310849 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-a8ae4168-8d0b-4cd2-aaf3-98ed335cd089 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5227b74d457640a4b4db8346ae48e388'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:49.330619 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-a8ae4168-8d0b-4cd2-aaf3-98ed335cd089 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5227b74d457640a4b4db8346ae48e388'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:49.469242 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-a8ae4168-8d0b-4cd2-aaf3-98ed335cd089 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5227b74d457640a4b4db8346ae48e388'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:49.470329 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-a8ae4168-8d0b-4cd2-aaf3-98ed335cd089 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5227b74d457640a4b4db8346ae48e388'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:49.478889 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-a8ae4168-8d0b-4cd2-aaf3-98ed335cd089 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5227b74d457640a4b4db8346ae48e388'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:49.497674 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 959/1895] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 24 17:26:48 2022] DELETE /identity/v3/users/5227b74d457640a4b4db8346ae48e388 => generated 0 bytes in 561 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:26:49.505300 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7125d7-bb70-49d9-9583-5f7db4da7a5d None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:49.506087 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7125d7-bb70-49d9-9583-5f7db4da7a5d None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:49.507711 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7125d7-bb70-49d9-9583-5f7db4da7a5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:49.539673 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-ae7125d7-bb70-49d9-9583-5f7db4da7a5d None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:26:49.540918 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-ae7125d7-bb70-49d9-9583-5f7db4da7a5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:26:49.586196 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:26:49.695456 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 937/1896] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:26:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:26:49.707315 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ae4168-8d0b-4cd2-aaf3-98ed335cd089 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:49.709546 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-a8ae4168-8d0b-4cd2-aaf3-98ed335cd089 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:26:49.801932 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ae4168-8d0b-4cd2-aaf3-98ed335cd089 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:49.808842 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbdb5cc-9ae2-4c5d-a9f3-e70c2db18fd6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e9ed4d47aef4140bfb2d3bb798cc559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:49.809734 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbdb5cc-9ae2-4c5d-a9f3-e70c2db18fd6 service neutron] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:49.810235 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbdb5cc-9ae2-4c5d-a9f3-e70c2db18fd6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:49.810883 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbdb5cc-9ae2-4c5d-a9f3-e70c2db18fd6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:49.823189 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbdb5cc-9ae2-4c5d-a9f3-e70c2db18fd6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:49.825905 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbdb5cc-9ae2-4c5d-a9f3-e70c2db18fd6 service neutron] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:49.856858 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 960/1897] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:26:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:26:50.639344 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7125d7-bb70-49d9-9583-5f7db4da7a5d None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:50.641579 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-ae7125d7-bb70-49d9-9583-5f7db4da7a5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:26:50.662228 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7125d7-bb70-49d9-9583-5f7db4da7a5d None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:50.672302 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1b3afd-51e6-4b60-b048-e48807fff6a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:50.673289 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1b3afd-51e6-4b60-b048-e48807fff6a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:50.673777 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1b3afd-51e6-4b60-b048-e48807fff6a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:50.674324 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1b3afd-51e6-4b60-b048-e48807fff6a8 admin admin] PATH_INFO: `/v3/projects/5c0c4ff4b452459980ef8067737854bb` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:50.676812 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1b3afd-51e6-4b60-b048-e48807fff6a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c0c4ff4b452459980ef8067737854bb)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:50.680563 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1b3afd-51e6-4b60-b048-e48807fff6a8 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:50.893555 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-5c1b3afd-51e6-4b60-b048-e48807fff6a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c0c4ff4b452459980ef8067737854bb', 'request_id': 'req-5c1b3afd-51e6-4b60-b048-e48807fff6a8'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:51.017529 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-5c1b3afd-51e6-4b60-b048-e48807fff6a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c0c4ff4b452459980ef8067737854bb', 'request_id': 'req-5c1b3afd-51e6-4b60-b048-e48807fff6a8'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:51.032361 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-5c1b3afd-51e6-4b60-b048-e48807fff6a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c0c4ff4b452459980ef8067737854bb', 'request_id': 'req-5c1b3afd-51e6-4b60-b048-e48807fff6a8'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:51.057995 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-5c1b3afd-51e6-4b60-b048-e48807fff6a8 admin admin] The token cache is being invalidate because project 5c0c4ff4b452459980ef8067737854bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81732) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:26:51.058726 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-5c1b3afd-51e6-4b60-b048-e48807fff6a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:51.060072 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-5c1b3afd-51e6-4b60-b048-e48807fff6a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:51.063974 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 938/1898] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 24 17:26:50 2022] DELETE /identity/v3/projects/5c0c4ff4b452459980ef8067737854bb => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:26:51.067996 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbdb5cc-9ae2-4c5d-a9f3-e70c2db18fd6 service neutron] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:51.080259 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbdb5cc-9ae2-4c5d-a9f3-e70c2db18fd6 service neutron] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:51.177500 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ee97b2-6333-49ff-9110-a9c3c991f55f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:51.178832 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ee97b2-6333-49ff-9110-a9c3c991f55f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:51.179670 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ee97b2-6333-49ff-9110-a9c3c991f55f admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:51.180151 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ee97b2-6333-49ff-9110-a9c3c991f55f admin admin] PATH_INFO: `/v3/users/e0f60d59ab7641c89397b055c2b5f86b` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:51.182395 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ee97b2-6333-49ff-9110-a9c3c991f55f admin admin] RBAC: Authorizing `identity:delete_user(user_id=e0f60d59ab7641c89397b055c2b5f86b)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:51.186106 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ee97b2-6333-49ff-9110-a9c3c991f55f admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:51.440600 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-76ee97b2-6333-49ff-9110-a9c3c991f55f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e0f60d59ab7641c89397b055c2b5f86b'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:51.455110 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:26:51.457863 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-76ee97b2-6333-49ff-9110-a9c3c991f55f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e0f60d59ab7641c89397b055c2b5f86b'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:51.523174 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-76ee97b2-6333-49ff-9110-a9c3c991f55f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e0f60d59ab7641c89397b055c2b5f86b'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:51.524347 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-76ee97b2-6333-49ff-9110-a9c3c991f55f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e0f60d59ab7641c89397b055c2b5f86b'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:51.540664 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-76ee97b2-6333-49ff-9110-a9c3c991f55f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e0f60d59ab7641c89397b055c2b5f86b'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:51.563334 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 961/1899] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 24 17:26:51 2022] DELETE /identity/v3/users/e0f60d59ab7641c89397b055c2b5f86b => generated 0 bytes in 492 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:26:52.353217 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1b3afd-51e6-4b60-b048-e48807fff6a8 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:52.356603 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-5c1b3afd-51e6-4b60-b048-e48807fff6a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:26:52.395403 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1b3afd-51e6-4b60-b048-e48807fff6a8 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:52.513450 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87dbdfc-4366-4018-a653-d9dc30ac3e85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:52.514482 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87dbdfc-4366-4018-a653-d9dc30ac3e85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:52.514993 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87dbdfc-4366-4018-a653-d9dc30ac3e85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:52.515460 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87dbdfc-4366-4018-a653-d9dc30ac3e85 admin admin] PATH_INFO: `/v3/projects/9d3d20b42a494a2185bed9102ecef015` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:52.517545 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87dbdfc-4366-4018-a653-d9dc30ac3e85 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d3d20b42a494a2185bed9102ecef015)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:52.523351 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87dbdfc-4366-4018-a653-d9dc30ac3e85 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:52.691199 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-e87dbdfc-4366-4018-a653-d9dc30ac3e85 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d3d20b42a494a2185bed9102ecef015', 'request_id': 'req-e87dbdfc-4366-4018-a653-d9dc30ac3e85'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:52.696177 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:26:52.777573 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-e87dbdfc-4366-4018-a653-d9dc30ac3e85 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d3d20b42a494a2185bed9102ecef015', 'request_id': 'req-e87dbdfc-4366-4018-a653-d9dc30ac3e85'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:52.787146 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-e87dbdfc-4366-4018-a653-d9dc30ac3e85 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d3d20b42a494a2185bed9102ecef015', 'request_id': 'req-e87dbdfc-4366-4018-a653-d9dc30ac3e85'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:52.805290 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-e87dbdfc-4366-4018-a653-d9dc30ac3e85 admin admin] The token cache is being invalidate because project 9d3d20b42a494a2185bed9102ecef015 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81732) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:26:52.809173 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-e87dbdfc-4366-4018-a653-d9dc30ac3e85 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:52.809173 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-e87dbdfc-4366-4018-a653-d9dc30ac3e85 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:52.816993 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 939/1900] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 24 17:26:52 2022] DELETE /identity/v3/projects/9d3d20b42a494a2185bed9102ecef015 => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:26:52.819443 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a911340-7fb4-499f-bc8f-d7e408fc1c1a None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:52.820183 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a911340-7fb4-499f-bc8f-d7e408fc1c1a None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:52.820563 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a911340-7fb4-499f-bc8f-d7e408fc1c1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:52.855783 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-6a911340-7fb4-499f-bc8f-d7e408fc1c1a None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:26:52.857857 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-6a911340-7fb4-499f-bc8f-d7e408fc1c1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:26:53.003706 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 962/1901] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:26:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:26:53.018006 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba9138e-f012-4872-84fa-9378a6a7d4a6 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:53.018466 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba9138e-f012-4872-84fa-9378a6a7d4a6 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:53.019024 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba9138e-f012-4872-84fa-9378a6a7d4a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:53.043075 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-8ba9138e-f012-4872-84fa-9378a6a7d4a6 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:26:53.045505 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-8ba9138e-f012-4872-84fa-9378a6a7d4a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:26:53.066359 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 940/1902] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:26:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:26:53.069069 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a911340-7fb4-499f-bc8f-d7e408fc1c1a None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:53.087147 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a911340-7fb4-499f-bc8f-d7e408fc1c1a None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:53.093837 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699e935c-dd55-4cff-8e91-0ad1782fbb32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:53.094596 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699e935c-dd55-4cff-8e91-0ad1782fbb32 admin admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:53.095007 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699e935c-dd55-4cff-8e91-0ad1782fbb32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:53.095734 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699e935c-dd55-4cff-8e91-0ad1782fbb32 admin admin] PATH_INFO: `/v3/domains` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:53.096546 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699e935c-dd55-4cff-8e91-0ad1782fbb32 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:26:53.096936 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699e935c-dd55-4cff-8e91-0ad1782fbb32 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:53.098868 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699e935c-dd55-4cff-8e91-0ad1782fbb32 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:53.117453 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 963/1903] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 24 17:26:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:26:53.119756 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba9138e-f012-4872-84fa-9378a6a7d4a6 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:53.139546 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba9138e-f012-4872-84fa-9378a6a7d4a6 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:53.148440 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7620be8e-acf7-4585-a332-9b77d39653fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:53.149997 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7620be8e-acf7-4585-a332-9b77d39653fc admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:53.150480 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7620be8e-acf7-4585-a332-9b77d39653fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:53.151634 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7620be8e-acf7-4585-a332-9b77d39653fc admin admin] PATH_INFO: `/v3/projects` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:53.157548 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7620be8e-acf7-4585-a332-9b77d39653fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:53.157548 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7620be8e-acf7-4585-a332-9b77d39653fc admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:53.275280 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 941/1904] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 24 17:26:53 2022] POST /identity/v3/projects => generated 385 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:26:53.280593 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699e935c-dd55-4cff-8e91-0ad1782fbb32 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:53.309396 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699e935c-dd55-4cff-8e91-0ad1782fbb32 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:53.438239 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5e6558-127e-4a80-91ac-84c01c236be4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:53.439926 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5e6558-127e-4a80-91ac-84c01c236be4 admin admin] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:53.440744 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5e6558-127e-4a80-91ac-84c01c236be4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:53.442023 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5e6558-127e-4a80-91ac-84c01c236be4 admin admin] PATH_INFO: `/v3/users` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:53.443590 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5e6558-127e-4a80-91ac-84c01c236be4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:53.451187 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5e6558-127e-4a80-91ac-84c01c236be4 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:53.594287 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 964/1905] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 24 17:26:53 2022] POST /identity/v3/users => generated 293 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:26:53.599388 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7620be8e-acf7-4585-a332-9b77d39653fc admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:53.619125 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7620be8e-acf7-4585-a332-9b77d39653fc admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:53.628139 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a668f867-e890-4a21-b3c9-ef2d513e4ed1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:53.629343 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a668f867-e890-4a21-b3c9-ef2d513e4ed1 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:53.629948 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a668f867-e890-4a21-b3c9-ef2d513e4ed1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:53.630552 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a668f867-e890-4a21-b3c9-ef2d513e4ed1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:53.632033 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a668f867-e890-4a21-b3c9-ef2d513e4ed1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:53.635400 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a668f867-e890-4a21-b3c9-ef2d513e4ed1 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:53.667384 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 942/1906] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:26:53 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:26:53.669050 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5e6558-127e-4a80-91ac-84c01c236be4 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:53.700530 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5e6558-127e-4a80-91ac-84c01c236be4 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:53.707345 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a716ed-cc27-460b-a6ae-581cef5a555a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:53.709085 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a716ed-cc27-460b-a6ae-581cef5a555a admin admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:53.709894 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a716ed-cc27-460b-a6ae-581cef5a555a admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:53.710759 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a716ed-cc27-460b-a6ae-581cef5a555a admin admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:53.713997 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a716ed-cc27-460b-a6ae-581cef5a555a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:53.717886 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a716ed-cc27-460b-a6ae-581cef5a555a admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:53.738222 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 965/1907] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:26:53 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:26:53.743368 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a668f867-e890-4a21-b3c9-ef2d513e4ed1 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:53.773180 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a668f867-e890-4a21-b3c9-ef2d513e4ed1 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:53.796821 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5470d851-61d2-4d25-9265-ad0043f3c351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:53.797997 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5470d851-61d2-4d25-9265-ad0043f3c351 admin admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:53.798507 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5470d851-61d2-4d25-9265-ad0043f3c351 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:53.799030 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5470d851-61d2-4d25-9265-ad0043f3c351 admin admin] PATH_INFO: `/v3/projects/fcb3189f96454148b6502dae69fbfdbf/users/f8070ad722c3465b8740fbe8cc968dd4/roles/60dc7e16aee746d5a3e8aa27d7d8525c` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:53.892993 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5470d851-61d2-4d25-9265-ad0043f3c351 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fcb3189f96454148b6502dae69fbfdbf, user_id=f8070ad722c3465b8740fbe8cc968dd4, role_id=60dc7e16aee746d5a3e8aa27d7d8525c)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:53.925251 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5470d851-61d2-4d25-9265-ad0043f3c351 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:54.048107 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c17a06-ba61-4fcc-8d95-2f5127dcf364 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:54.048806 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 943/1908] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 24 17:26:53 2022] PUT /identity/v3/projects/fcb3189f96454148b6502dae69fbfdbf/users/f8070ad722c3465b8740fbe8cc968dd4/roles/60dc7e16aee746d5a3e8aa27d7d8525c => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:26:54.048960 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c17a06-ba61-4fcc-8d95-2f5127dcf364 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:54.049322 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c17a06-ba61-4fcc-8d95-2f5127dcf364 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:54.151494 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-a1c17a06-ba61-4fcc-8d95-2f5127dcf364 None None] MFA Rules not processed for user `f8070ad722c3465b8740fbe8cc968dd4`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:26:54.152853 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-a1c17a06-ba61-4fcc-8d95-2f5127dcf364 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:26:54.233904 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:26:54.284504 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5470d851-61d2-4d25-9265-ad0043f3c351 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:54.295705 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 966/1909] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:26:54 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:26:54.384277 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5470d851-61d2-4d25-9265-ad0043f3c351 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:54.392494 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be107880-6993-4853-bec3-a0b1958777d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:54.393454 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be107880-6993-4853-bec3-a0b1958777d4 admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:54.393949 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be107880-6993-4853-bec3-a0b1958777d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:54.394448 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be107880-6993-4853-bec3-a0b1958777d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:54.399013 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be107880-6993-4853-bec3-a0b1958777d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:54.403333 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be107880-6993-4853-bec3-a0b1958777d4 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:54.524098 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 944/1910] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 24 17:26:54 2022] POST /identity/v3/projects => generated 383 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:26:54.530439 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c17a06-ba61-4fcc-8d95-2f5127dcf364 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:54.542669 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c17a06-ba61-4fcc-8d95-2f5127dcf364 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:54.650738 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4781eff9-2d46-42ed-b2a8-6105780617f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:54.652325 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4781eff9-2d46-42ed-b2a8-6105780617f0 admin admin] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:54.652780 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4781eff9-2d46-42ed-b2a8-6105780617f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:54.653148 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4781eff9-2d46-42ed-b2a8-6105780617f0 admin admin] PATH_INFO: `/v3/users` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:54.654182 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4781eff9-2d46-42ed-b2a8-6105780617f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:54.656987 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4781eff9-2d46-42ed-b2a8-6105780617f0 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:54.812901 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 967/1911] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 24 17:26:54 2022] POST /identity/v3/users => generated 292 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:26:54.814526 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be107880-6993-4853-bec3-a0b1958777d4 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:54.841365 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be107880-6993-4853-bec3-a0b1958777d4 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:54.850877 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033919f2-e17b-4095-80d7-5a549a4b6752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:54.851237 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033919f2-e17b-4095-80d7-5a549a4b6752 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:54.851726 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033919f2-e17b-4095-80d7-5a549a4b6752 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:54.854955 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033919f2-e17b-4095-80d7-5a549a4b6752 admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:54.854955 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033919f2-e17b-4095-80d7-5a549a4b6752 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:54.857381 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033919f2-e17b-4095-80d7-5a549a4b6752 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:54.893399 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4781eff9-2d46-42ed-b2a8-6105780617f0 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:54.902164 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 945/1912] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:26:54 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:26:54.915644 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4781eff9-2d46-42ed-b2a8-6105780617f0 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:54.925212 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d471391-797a-4f18-aa3f-e5dd048727a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:54.927133 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d471391-797a-4f18-aa3f-e5dd048727a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:54.927133 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d471391-797a-4f18-aa3f-e5dd048727a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:54.927953 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d471391-797a-4f18-aa3f-e5dd048727a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:54.929356 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d471391-797a-4f18-aa3f-e5dd048727a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:54.933106 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d471391-797a-4f18-aa3f-e5dd048727a4 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:54.956842 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 968/1913] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:26:54 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:26:54.965359 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033919f2-e17b-4095-80d7-5a549a4b6752 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:54.985054 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033919f2-e17b-4095-80d7-5a549a4b6752 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:54.995442 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7b5d05-69c2-495d-a98e-3d55fab38884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:54.996821 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7b5d05-69c2-495d-a98e-3d55fab38884 admin admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:54.997387 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7b5d05-69c2-495d-a98e-3d55fab38884 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:54.997941 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7b5d05-69c2-495d-a98e-3d55fab38884 admin admin] PATH_INFO: `/v3/projects/71440c8b913747ec97420968d48b753a/users/99d7fdad92544e38877fff5bfccc56ba/roles/60dc7e16aee746d5a3e8aa27d7d8525c` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:55.034076 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7b5d05-69c2-495d-a98e-3d55fab38884 admin admin] RBAC: Authorizing `identity:create_grant(project_id=71440c8b913747ec97420968d48b753a, user_id=99d7fdad92544e38877fff5bfccc56ba, role_id=60dc7e16aee746d5a3e8aa27d7d8525c)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:55.049702 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7b5d05-69c2-495d-a98e-3d55fab38884 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:55.193660 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 946/1914] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 24 17:26:54 2022] PUT /identity/v3/projects/71440c8b913747ec97420968d48b753a/users/99d7fdad92544e38877fff5bfccc56ba/roles/60dc7e16aee746d5a3e8aa27d7d8525c => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:26:55.201474 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc21434a-6aaf-45d9-af09-695269b23f53 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:55.202357 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc21434a-6aaf-45d9-af09-695269b23f53 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:55.202725 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc21434a-6aaf-45d9-af09-695269b23f53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:55.296668 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-fc21434a-6aaf-45d9-af09-695269b23f53 None None] MFA Rules not processed for user `99d7fdad92544e38877fff5bfccc56ba`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:26:55.300991 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-fc21434a-6aaf-45d9-af09-695269b23f53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:26:55.452699 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 969/1915] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:26:55 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:26:55.457726 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7b5d05-69c2-495d-a98e-3d55fab38884 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:55.597479 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:26:55.693064 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7b5d05-69c2-495d-a98e-3d55fab38884 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:55.712458 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b99926f-6333-4c4d-8f71-08389cc17ea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:55.713701 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b99926f-6333-4c4d-8f71-08389cc17ea9 admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:55.714328 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b99926f-6333-4c4d-8f71-08389cc17ea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:55.715018 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b99926f-6333-4c4d-8f71-08389cc17ea9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:55.721393 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b99926f-6333-4c4d-8f71-08389cc17ea9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:55.722978 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b99926f-6333-4c4d-8f71-08389cc17ea9 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:55.834362 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 947/1916] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 24 17:26:55 2022] POST /identity/v3/projects => generated 383 bytes in 380 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:26:55.836370 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc21434a-6aaf-45d9-af09-695269b23f53 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:55.856943 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc21434a-6aaf-45d9-af09-695269b23f53 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:55.932881 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:26:55.962374 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4088ca8f-9a81-41e3-b6f5-d4c47c4676d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:55.964020 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4088ca8f-9a81-41e3-b6f5-d4c47c4676d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:55.964424 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4088ca8f-9a81-41e3-b6f5-d4c47c4676d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:55.964871 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4088ca8f-9a81-41e3-b6f5-d4c47c4676d6 admin admin] PATH_INFO: `/v3/users` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:55.965915 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4088ca8f-9a81-41e3-b6f5-d4c47c4676d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:55.969459 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4088ca8f-9a81-41e3-b6f5-d4c47c4676d6 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:56.091854 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 970/1917] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 24 17:26:55 2022] POST /identity/v3/users => generated 298 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:26:56.097009 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b99926f-6333-4c4d-8f71-08389cc17ea9 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:56.124674 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b99926f-6333-4c4d-8f71-08389cc17ea9 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:56.134839 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee82dd4a-451a-46df-803b-7a2d8d957766 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:56.137296 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee82dd4a-451a-46df-803b-7a2d8d957766 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:56.137782 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee82dd4a-451a-46df-803b-7a2d8d957766 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:56.138268 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee82dd4a-451a-46df-803b-7a2d8d957766 admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:56.139488 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee82dd4a-451a-46df-803b-7a2d8d957766 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:56.142799 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee82dd4a-451a-46df-803b-7a2d8d957766 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:56.163679 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 948/1918] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:26:56 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:26:56.168826 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4088ca8f-9a81-41e3-b6f5-d4c47c4676d6 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:56.196110 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4088ca8f-9a81-41e3-b6f5-d4c47c4676d6 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:56.208380 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26c0374-d610-4a06-a525-dca96e7a05eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:56.210154 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26c0374-d610-4a06-a525-dca96e7a05eb admin admin] REQUEST_METHOD: `PUT` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:56.210154 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26c0374-d610-4a06-a525-dca96e7a05eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:56.210613 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26c0374-d610-4a06-a525-dca96e7a05eb admin admin] PATH_INFO: `/v3/projects/a5862a12b0b74b8fb05b0d7edff67dcd/users/53a4e528b55e49dfba20fad2425575c2/roles/c5346bc54abf445f9900edf94264c850` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:56.233332 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26c0374-d610-4a06-a525-dca96e7a05eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5862a12b0b74b8fb05b0d7edff67dcd, user_id=53a4e528b55e49dfba20fad2425575c2, role_id=c5346bc54abf445f9900edf94264c850)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:56.236155 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26c0374-d610-4a06-a525-dca96e7a05eb admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:56.468477 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 971/1919] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 24 17:26:56 2022] PUT /identity/v3/projects/a5862a12b0b74b8fb05b0d7edff67dcd/users/53a4e528b55e49dfba20fad2425575c2/roles/c5346bc54abf445f9900edf94264c850 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:26:56.469902 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee82dd4a-451a-46df-803b-7a2d8d957766 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:56.488957 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee82dd4a-451a-46df-803b-7a2d8d957766 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:56.596314 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea375a65-520b-4036-a554-cc75e98ec790 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:56.597032 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea375a65-520b-4036-a554-cc75e98ec790 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:56.597413 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea375a65-520b-4036-a554-cc75e98ec790 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:56.597686 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea375a65-520b-4036-a554-cc75e98ec790 admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:56.598361 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea375a65-520b-4036-a554-cc75e98ec790 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:56.600069 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea375a65-520b-4036-a554-cc75e98ec790 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:56.613529 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 949/1920] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:26:56 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:26:56.620188 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26c0374-d610-4a06-a525-dca96e7a05eb admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:56.643490 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26c0374-d610-4a06-a525-dca96e7a05eb admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:56.653154 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea6dbc1-3d38-4dcf-8118-d48af84d5015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:56.654319 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea6dbc1-3d38-4dcf-8118-d48af84d5015 admin admin] REQUEST_METHOD: `PUT` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:56.654868 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea6dbc1-3d38-4dcf-8118-d48af84d5015 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:56.655411 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea6dbc1-3d38-4dcf-8118-d48af84d5015 admin admin] PATH_INFO: `/v3/projects/a5862a12b0b74b8fb05b0d7edff67dcd/users/53a4e528b55e49dfba20fad2425575c2/roles/c5346bc54abf445f9900edf94264c850` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:56.658848 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea6dbc1-3d38-4dcf-8118-d48af84d5015 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5862a12b0b74b8fb05b0d7edff67dcd, user_id=53a4e528b55e49dfba20fad2425575c2, role_id=c5346bc54abf445f9900edf94264c850)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:56.664387 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea6dbc1-3d38-4dcf-8118-d48af84d5015 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:56.682590 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 972/1921] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 24 17:26:56 2022] PUT /identity/v3/projects/a5862a12b0b74b8fb05b0d7edff67dcd/users/53a4e528b55e49dfba20fad2425575c2/roles/c5346bc54abf445f9900edf94264c850 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:26:56.703295 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99ac901-8fad-426d-926c-d7b665d492ad None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:56.703752 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99ac901-8fad-426d-926c-d7b665d492ad None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:56.704188 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99ac901-8fad-426d-926c-d7b665d492ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:56.753095 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-b99ac901-8fad-426d-926c-d7b665d492ad None None] MFA Rules not processed for user `53a4e528b55e49dfba20fad2425575c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:26:56.754917 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-b99ac901-8fad-426d-926c-d7b665d492ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:26:56.846828 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 950/1922] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:26:56 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:26:56.863465 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea6dbc1-3d38-4dcf-8118-d48af84d5015 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:56.866078 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-4ea6dbc1-3d38-4dcf-8118-d48af84d5015 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:26:57.000251 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea6dbc1-3d38-4dcf-8118-d48af84d5015 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:57.006542 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15f7701-608d-42f2-be52-d395d1dc5ce0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be06246d47584b529150ceb98a5cfd95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:57.007650 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15f7701-608d-42f2-be52-d395d1dc5ce0 service cinder] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:57.011011 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15f7701-608d-42f2-be52-d395d1dc5ce0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:57.011011 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15f7701-608d-42f2-be52-d395d1dc5ce0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:57.138962 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15f7701-608d-42f2-be52-d395d1dc5ce0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:57.140489 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15f7701-608d-42f2-be52-d395d1dc5ce0 service cinder] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:57.154695 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 973/1923] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:26:56 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:26:59.075518 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99ac901-8fad-426d-926c-d7b665d492ad None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:26:59.077744 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-b99ac901-8fad-426d-926c-d7b665d492ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:26:59.132575 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:26:59.199499 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99ac901-8fad-426d-926c-d7b665d492ad None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:26:59.208438 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b91df21-4e87-4999-bc6a-29923a83daa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:26:59.209832 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b91df21-4e87-4999-bc6a-29923a83daa6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:26:59.210058 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b91df21-4e87-4999-bc6a-29923a83daa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:26:59.210319 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b91df21-4e87-4999-bc6a-29923a83daa6 admin admin] PATH_INFO: `/v3/users/427a581c9c47411db485028e40830208` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:26:59.211590 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b91df21-4e87-4999-bc6a-29923a83daa6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=427a581c9c47411db485028e40830208)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:26:59.213691 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b91df21-4e87-4999-bc6a-29923a83daa6 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:26:59.614509 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-1b91df21-4e87-4999-bc6a-29923a83daa6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '427a581c9c47411db485028e40830208'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:59.849804 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-1b91df21-4e87-4999-bc6a-29923a83daa6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '427a581c9c47411db485028e40830208'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:59.851953 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-1b91df21-4e87-4999-bc6a-29923a83daa6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '427a581c9c47411db485028e40830208'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:59.861430 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-1b91df21-4e87-4999-bc6a-29923a83daa6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '427a581c9c47411db485028e40830208'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:59.862185 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-1b91df21-4e87-4999-bc6a-29923a83daa6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '427a581c9c47411db485028e40830208'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:26:59.893001 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 951/1924] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 24 17:26:59 2022] DELETE /identity/v3/users/427a581c9c47411db485028e40830208 => generated 0 bytes in 807 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:27:00.253612 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15f7701-608d-42f2-be52-d395d1dc5ce0 service cinder] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:00.285981 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15f7701-608d-42f2-be52-d395d1dc5ce0 service cinder] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:00.380236 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:00.398241 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb03629-115b-44b0-b496-e558afa4c6ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:00.399336 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb03629-115b-44b0-b496-e558afa4c6ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:00.399828 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb03629-115b-44b0-b496-e558afa4c6ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:00.400312 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb03629-115b-44b0-b496-e558afa4c6ae admin admin] PATH_INFO: `/v3/projects/bb4465ad66dc4fdca7943fd07cb82807` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:00.403125 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb03629-115b-44b0-b496-e558afa4c6ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb4465ad66dc4fdca7943fd07cb82807)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:00.406785 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb03629-115b-44b0-b496-e558afa4c6ae admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:00.677000 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-adb03629-115b-44b0-b496-e558afa4c6ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb4465ad66dc4fdca7943fd07cb82807', 'request_id': 'req-adb03629-115b-44b0-b496-e558afa4c6ae'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:27:00.833830 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-adb03629-115b-44b0-b496-e558afa4c6ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb4465ad66dc4fdca7943fd07cb82807', 'request_id': 'req-adb03629-115b-44b0-b496-e558afa4c6ae'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:27:00.855766 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-adb03629-115b-44b0-b496-e558afa4c6ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb4465ad66dc4fdca7943fd07cb82807', 'request_id': 'req-adb03629-115b-44b0-b496-e558afa4c6ae'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:27:00.859264 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-adb03629-115b-44b0-b496-e558afa4c6ae admin admin] The token cache is being invalidate because project bb4465ad66dc4fdca7943fd07cb82807 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81731) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:27:00.859780 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-adb03629-115b-44b0-b496-e558afa4c6ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:27:00.860874 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-adb03629-115b-44b0-b496-e558afa4c6ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:27:00.865646 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 974/1925] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 24 17:27:00 2022] DELETE /identity/v3/projects/bb4465ad66dc4fdca7943fd07cb82807 => generated 0 bytes in 610 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:27:00.867089 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b91df21-4e87-4999-bc6a-29923a83daa6 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:00.869276 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-1b91df21-4e87-4999-bc6a-29923a83daa6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:27:00.874028 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb03629-115b-44b0-b496-e558afa4c6ae admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:00.877313 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-adb03629-115b-44b0-b496-e558afa4c6ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:27:00.892697 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b91df21-4e87-4999-bc6a-29923a83daa6 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:01.003697 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4620b08-8a3b-4071-b1d4-d272a9f82187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:01.004088 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4620b08-8a3b-4071-b1d4-d272a9f82187 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:01.004414 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4620b08-8a3b-4071-b1d4-d272a9f82187 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:01.004694 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4620b08-8a3b-4071-b1d4-d272a9f82187 admin admin] PATH_INFO: `/v3/users/8b1733d114e84704ad599efcc7b4c7be` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:01.005873 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4620b08-8a3b-4071-b1d4-d272a9f82187 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b1733d114e84704ad599efcc7b4c7be)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:01.007860 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4620b08-8a3b-4071-b1d4-d272a9f82187 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:01.009160 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb03629-115b-44b0-b496-e558afa4c6ae admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:01.015183 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f034f2-1297-45eb-adf4-d3ade15caad4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be06246d47584b529150ceb98a5cfd95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:01.016605 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f034f2-1297-45eb-adf4-d3ade15caad4 service cinder] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:01.017192 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f034f2-1297-45eb-adf4-d3ade15caad4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:01.017769 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f034f2-1297-45eb-adf4-d3ade15caad4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:01.020749 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-49f034f2-1297-45eb-adf4-d3ade15caad4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:27:01.117101 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f034f2-1297-45eb-adf4-d3ade15caad4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:01.119512 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f034f2-1297-45eb-adf4-d3ade15caad4 service cinder] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:01.134777 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 975/1926] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:27:00 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:27:01.342575 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:01.345692 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-e4620b08-8a3b-4071-b1d4-d272a9f82187 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b1733d114e84704ad599efcc7b4c7be'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:27:01.509341 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-e4620b08-8a3b-4071-b1d4-d272a9f82187 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b1733d114e84704ad599efcc7b4c7be'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:27:01.511255 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-e4620b08-8a3b-4071-b1d4-d272a9f82187 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b1733d114e84704ad599efcc7b4c7be'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:27:01.537812 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-e4620b08-8a3b-4071-b1d4-d272a9f82187 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b1733d114e84704ad599efcc7b4c7be'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:27:01.540644 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-e4620b08-8a3b-4071-b1d4-d272a9f82187 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b1733d114e84704ad599efcc7b4c7be'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:27:01.581688 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 952/1927] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 24 17:27:00 2022] DELETE /identity/v3/users/8b1733d114e84704ad599efcc7b4c7be => generated 0 bytes in 711 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:27:02.293886 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f034f2-1297-45eb-adf4-d3ade15caad4 service cinder] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:02.296597 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-49f034f2-1297-45eb-adf4-d3ade15caad4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:27:02.323484 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f034f2-1297-45eb-adf4-d3ade15caad4 service cinder] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:02.371502 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:02.407153 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e03889a-2e8f-450a-ae95-bc70c9dcc869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:02.419933 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e03889a-2e8f-450a-ae95-bc70c9dcc869 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:02.419933 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e03889a-2e8f-450a-ae95-bc70c9dcc869 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:02.419933 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e03889a-2e8f-450a-ae95-bc70c9dcc869 admin admin] PATH_INFO: `/v3/projects/3d3d9d27fbbd48549a012a01bc4392e2` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:02.419933 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e03889a-2e8f-450a-ae95-bc70c9dcc869 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d3d9d27fbbd48549a012a01bc4392e2)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:02.419933 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e03889a-2e8f-450a-ae95-bc70c9dcc869 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:02.585246 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-0e03889a-2e8f-450a-ae95-bc70c9dcc869 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d3d9d27fbbd48549a012a01bc4392e2', 'request_id': 'req-0e03889a-2e8f-450a-ae95-bc70c9dcc869'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:27:02.709369 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-0e03889a-2e8f-450a-ae95-bc70c9dcc869 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d3d9d27fbbd48549a012a01bc4392e2', 'request_id': 'req-0e03889a-2e8f-450a-ae95-bc70c9dcc869'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:27:02.720743 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-0e03889a-2e8f-450a-ae95-bc70c9dcc869 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d3d9d27fbbd48549a012a01bc4392e2', 'request_id': 'req-0e03889a-2e8f-450a-ae95-bc70c9dcc869'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:27:02.739520 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-0e03889a-2e8f-450a-ae95-bc70c9dcc869 admin admin] The token cache is being invalidate because project 3d3d9d27fbbd48549a012a01bc4392e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81731) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:27:02.740190 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-0e03889a-2e8f-450a-ae95-bc70c9dcc869 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:27:02.741808 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-0e03889a-2e8f-450a-ae95-bc70c9dcc869 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:27:02.743851 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 976/1928] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 24 17:27:02 2022] DELETE /identity/v3/projects/3d3d9d27fbbd48549a012a01bc4392e2 => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:27:02.751374 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96adac48-850f-433f-8f56-5a09c8f9f935 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:02.751885 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96adac48-850f-433f-8f56-5a09c8f9f935 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:02.752177 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96adac48-850f-433f-8f56-5a09c8f9f935 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:02.778688 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-96adac48-850f-433f-8f56-5a09c8f9f935 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:27:02.779957 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-96adac48-850f-433f-8f56-5a09c8f9f935 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:27:02.874005 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 953/1929] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:27:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:27:02.887019 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ef3491-7af9-4192-8677-431b5f72b934 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:02.887513 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ef3491-7af9-4192-8677-431b5f72b934 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:02.887998 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ef3491-7af9-4192-8677-431b5f72b934 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:02.927321 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-e6ef3491-7af9-4192-8677-431b5f72b934 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:27:02.929455 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-e6ef3491-7af9-4192-8677-431b5f72b934 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:27:02.952222 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 977/1930] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:27:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:27:02.956918 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96adac48-850f-433f-8f56-5a09c8f9f935 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:02.979340 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96adac48-850f-433f-8f56-5a09c8f9f935 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:02.988939 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b49ecd1-d391-4fa0-862a-6a4dd112fb53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:02.989206 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b49ecd1-d391-4fa0-862a-6a4dd112fb53 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:02.989583 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b49ecd1-d391-4fa0-862a-6a4dd112fb53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:02.990086 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b49ecd1-d391-4fa0-862a-6a4dd112fb53 admin admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:02.991385 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b49ecd1-d391-4fa0-862a-6a4dd112fb53 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:27:02.991903 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b49ecd1-d391-4fa0-862a-6a4dd112fb53 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:02.993663 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b49ecd1-d391-4fa0-862a-6a4dd112fb53 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:03.007340 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 954/1931] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 24 17:27:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:27:03.014180 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ef3491-7af9-4192-8677-431b5f72b934 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:03.030683 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ef3491-7af9-4192-8677-431b5f72b934 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:03.039269 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dfbd2c-9db7-4c57-a308-3b21c5712a5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:03.040068 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dfbd2c-9db7-4c57-a308-3b21c5712a5e admin admin] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:03.040477 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dfbd2c-9db7-4c57-a308-3b21c5712a5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:03.040858 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dfbd2c-9db7-4c57-a308-3b21c5712a5e admin admin] PATH_INFO: `/v3/projects` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:03.041860 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9dfbd2c-9db7-4c57-a308-3b21c5712a5e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:03.044761 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9dfbd2c-9db7-4c57-a308-3b21c5712a5e admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:03.207845 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 978/1932] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 24 17:27:03 2022] POST /identity/v3/projects => generated 375 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:27:03.208759 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b49ecd1-d391-4fa0-862a-6a4dd112fb53 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:03.223276 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b49ecd1-d391-4fa0-862a-6a4dd112fb53 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:03.346804 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b9a8fd-a746-48eb-9a64-8b821a74d77e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:03.347158 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b9a8fd-a746-48eb-9a64-8b821a74d77e admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:03.347158 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b9a8fd-a746-48eb-9a64-8b821a74d77e admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:03.347508 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b9a8fd-a746-48eb-9a64-8b821a74d77e admin admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:03.348102 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b9a8fd-a746-48eb-9a64-8b821a74d77e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:03.350122 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b9a8fd-a746-48eb-9a64-8b821a74d77e admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:03.435424 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 955/1933] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 24 17:27:03 2022] POST /identity/v3/users => generated 288 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:27:03.441392 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dfbd2c-9db7-4c57-a308-3b21c5712a5e admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:03.475305 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dfbd2c-9db7-4c57-a308-3b21c5712a5e admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:03.475305 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4739d394-3c62-451d-86b5-c20e2b1c1993 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:03.475305 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4739d394-3c62-451d-86b5-c20e2b1c1993 admin admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:03.475305 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4739d394-3c62-451d-86b5-c20e2b1c1993 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:03.475809 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4739d394-3c62-451d-86b5-c20e2b1c1993 admin admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:03.475809 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4739d394-3c62-451d-86b5-c20e2b1c1993 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:03.477300 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4739d394-3c62-451d-86b5-c20e2b1c1993 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:03.494812 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 979/1934] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:27:03 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:27:03.495545 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b9a8fd-a746-48eb-9a64-8b821a74d77e admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:03.518472 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b9a8fd-a746-48eb-9a64-8b821a74d77e admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:03.525115 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b30361-3fcf-48a4-9431-c7778e753041 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:03.535081 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b30361-3fcf-48a4-9431-c7778e753041 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:03.535081 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b30361-3fcf-48a4-9431-c7778e753041 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:03.535081 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b30361-3fcf-48a4-9431-c7778e753041 admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:03.535081 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b30361-3fcf-48a4-9431-c7778e753041 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:03.535081 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b30361-3fcf-48a4-9431-c7778e753041 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:03.566971 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 956/1935] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:27:03 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:27:03.570003 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4739d394-3c62-451d-86b5-c20e2b1c1993 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:03.587414 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4739d394-3c62-451d-86b5-c20e2b1c1993 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:03.598100 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c50e7b-49bb-47e9-abf3-176c65c71f23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:03.599444 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c50e7b-49bb-47e9-abf3-176c65c71f23 admin admin] REQUEST_METHOD: `PUT` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:03.599867 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c50e7b-49bb-47e9-abf3-176c65c71f23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:03.600311 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c50e7b-49bb-47e9-abf3-176c65c71f23 admin admin] PATH_INFO: `/v3/projects/6159a8f79fc84643a62c1b12af19787a/users/f84b45bd1db04bc09f42cbf34a0ffd48/roles/60dc7e16aee746d5a3e8aa27d7d8525c` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:03.608511 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b30361-3fcf-48a4-9431-c7778e753041 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:03.611071 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-a6b30361-3fcf-48a4-9431-c7778e753041 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:27:03.624919 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c50e7b-49bb-47e9-abf3-176c65c71f23 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6159a8f79fc84643a62c1b12af19787a, user_id=f84b45bd1db04bc09f42cbf34a0ffd48, role_id=60dc7e16aee746d5a3e8aa27d7d8525c)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:03.630990 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c50e7b-49bb-47e9-abf3-176c65c71f23 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:03.662809 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:03.705210 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b30361-3fcf-48a4-9431-c7778e753041 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:03.709895 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d6a94f-9b87-4358-904e-d444020feeae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be06246d47584b529150ceb98a5cfd95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:03.710990 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d6a94f-9b87-4358-904e-d444020feeae service cinder] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:03.711496 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d6a94f-9b87-4358-904e-d444020feeae service cinder] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:03.711974 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d6a94f-9b87-4358-904e-d444020feeae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:03.714887 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-45d6a94f-9b87-4358-904e-d444020feeae service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:27:03.754236 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 980/1936] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 24 17:27:03 2022] PUT /identity/v3/projects/6159a8f79fc84643a62c1b12af19787a/users/f84b45bd1db04bc09f42cbf34a0ffd48/roles/60dc7e16aee746d5a3e8aa27d7d8525c => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:27:03.772331 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a337b0b-090f-4509-a67c-51ae0e542d5c None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:03.772820 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a337b0b-090f-4509-a67c-51ae0e542d5c None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:03.773304 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a337b0b-090f-4509-a67c-51ae0e542d5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:03.816625 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d6a94f-9b87-4358-904e-d444020feeae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:03.817385 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-7a337b0b-090f-4509-a67c-51ae0e542d5c None None] MFA Rules not processed for user `f84b45bd1db04bc09f42cbf34a0ffd48`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:27:03.818575 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-7a337b0b-090f-4509-a67c-51ae0e542d5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:27:03.819300 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d6a94f-9b87-4358-904e-d444020feeae service cinder] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:03.837924 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 957/1937] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:27:03 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:27:03.900021 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:03.920314 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 981/1938] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:27:03 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:27:03.938541 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d6a94f-9b87-4358-904e-d444020feeae service cinder] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:03.940777 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-45d6a94f-9b87-4358-904e-d444020feeae service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:27:03.958053 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d6a94f-9b87-4358-904e-d444020feeae service cinder] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:04.053494 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c34d5a-a649-4097-96c8-373730b8ea46 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be06246d47584b529150ceb98a5cfd95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:04.055706 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c34d5a-a649-4097-96c8-373730b8ea46 service cinder] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:04.056188 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c34d5a-a649-4097-96c8-373730b8ea46 service cinder] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:04.056670 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c34d5a-a649-4097-96c8-373730b8ea46 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:04.081086 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c34d5a-a649-4097-96c8-373730b8ea46 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:04.083406 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c34d5a-a649-4097-96c8-373730b8ea46 service cinder] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:04.097460 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 958/1939] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:27:03 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:27:14.090444 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=63 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:14.100016 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=5 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:20.934304 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a337b0b-090f-4509-a67c-51ae0e542d5c None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:21.029784 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a337b0b-090f-4509-a67c-51ae0e542d5c None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:21.035809 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a419c3d1-aab0-4197-8c23-44da2406a5a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:21.036570 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a419c3d1-aab0-4197-8c23-44da2406a5a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:21.036984 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a419c3d1-aab0-4197-8c23-44da2406a5a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:21.037347 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a419c3d1-aab0-4197-8c23-44da2406a5a7 admin admin] PATH_INFO: `/v3/users/f84b45bd1db04bc09f42cbf34a0ffd48` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:21.038623 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a419c3d1-aab0-4197-8c23-44da2406a5a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f84b45bd1db04bc09f42cbf34a0ffd48)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:21.040614 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a419c3d1-aab0-4197-8c23-44da2406a5a7 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:21.284841 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-a419c3d1-aab0-4197-8c23-44da2406a5a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f84b45bd1db04bc09f42cbf34a0ffd48'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:27:21.313525 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-a419c3d1-aab0-4197-8c23-44da2406a5a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f84b45bd1db04bc09f42cbf34a0ffd48'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:27:21.397468 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c34d5a-a649-4097-96c8-373730b8ea46 service cinder] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:21.399855 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-46c34d5a-a649-4097-96c8-373730b8ea46 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:27:21.408926 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-a419c3d1-aab0-4197-8c23-44da2406a5a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f84b45bd1db04bc09f42cbf34a0ffd48'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:27:21.408926 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-a419c3d1-aab0-4197-8c23-44da2406a5a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f84b45bd1db04bc09f42cbf34a0ffd48'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:27:21.417154 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-a419c3d1-aab0-4197-8c23-44da2406a5a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f84b45bd1db04bc09f42cbf34a0ffd48'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:27:21.423862 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c34d5a-a649-4097-96c8-373730b8ea46 service cinder] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:21.424087 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 982/1940] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 24 17:27:20 2022] DELETE /identity/v3/users/f84b45bd1db04bc09f42cbf34a0ffd48 => generated 0 bytes in 491 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:27:21.431032 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c44157-f4f6-4017-9948-94cfc784aba8 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:21.431540 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c44157-f4f6-4017-9948-94cfc784aba8 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:21.432049 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c44157-f4f6-4017-9948-94cfc784aba8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:21.468414 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-97c44157-f4f6-4017-9948-94cfc784aba8 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:27:21.470851 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-97c44157-f4f6-4017-9948-94cfc784aba8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:27:21.537452 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a751dba5-b223-457f-afb2-d626b9b5ef69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:21.538873 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a751dba5-b223-457f-afb2-d626b9b5ef69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:21.539079 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a751dba5-b223-457f-afb2-d626b9b5ef69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:21.539612 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a751dba5-b223-457f-afb2-d626b9b5ef69 admin admin] PATH_INFO: `/v3/users/f8070ad722c3465b8740fbe8cc968dd4` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:21.541739 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a751dba5-b223-457f-afb2-d626b9b5ef69 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f8070ad722c3465b8740fbe8cc968dd4)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:21.545155 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a751dba5-b223-457f-afb2-d626b9b5ef69 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:21.590664 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 983/1941] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:27:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:27:21.605680 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c44157-f4f6-4017-9948-94cfc784aba8 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:21.607537 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-97c44157-f4f6-4017-9948-94cfc784aba8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:27:21.649363 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:21.716378 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c44157-f4f6-4017-9948-94cfc784aba8 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:21.725155 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42d4436-cab3-46bc-a48b-4bf7bb74441f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e9ed4d47aef4140bfb2d3bb798cc559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:21.725465 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42d4436-cab3-46bc-a48b-4bf7bb74441f service neutron] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:21.725768 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42d4436-cab3-46bc-a48b-4bf7bb74441f service neutron] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:21.726298 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42d4436-cab3-46bc-a48b-4bf7bb74441f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:21.748393 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42d4436-cab3-46bc-a48b-4bf7bb74441f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:21.751735 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42d4436-cab3-46bc-a48b-4bf7bb74441f service neutron] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:21.776159 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 984/1942] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:27:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:27:21.794318 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-a751dba5-b223-457f-afb2-d626b9b5ef69 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f8070ad722c3465b8740fbe8cc968dd4'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:27:21.879163 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-a751dba5-b223-457f-afb2-d626b9b5ef69 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f8070ad722c3465b8740fbe8cc968dd4'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:27:21.879832 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-a751dba5-b223-457f-afb2-d626b9b5ef69 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f8070ad722c3465b8740fbe8cc968dd4'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:27:21.896794 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-a751dba5-b223-457f-afb2-d626b9b5ef69 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f8070ad722c3465b8740fbe8cc968dd4'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:27:21.899656 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-a751dba5-b223-457f-afb2-d626b9b5ef69 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f8070ad722c3465b8740fbe8cc968dd4'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:27:21.938524 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 959/1943] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 24 17:27:21 2022] DELETE /identity/v3/users/f8070ad722c3465b8740fbe8cc968dd4 => generated 0 bytes in 542 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:27:21.944499 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585a8485-a64a-4f35-88d4-582549c1504a None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:21.945192 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585a8485-a64a-4f35-88d4-582549c1504a None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:21.945696 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585a8485-a64a-4f35-88d4-582549c1504a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:21.977202 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-585a8485-a64a-4f35-88d4-582549c1504a None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:27:21.978550 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-585a8485-a64a-4f35-88d4-582549c1504a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:27:22.118322 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 985/1944] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:27:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:27:22.143527 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a751dba5-b223-457f-afb2-d626b9b5ef69 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:22.145472 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-a751dba5-b223-457f-afb2-d626b9b5ef69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:27:22.243136 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a751dba5-b223-457f-afb2-d626b9b5ef69 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:22.252139 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043f2288-1322-4ffc-9add-f2c5e891412c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e9ed4d47aef4140bfb2d3bb798cc559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:22.253149 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043f2288-1322-4ffc-9add-f2c5e891412c service neutron] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:22.253628 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043f2288-1322-4ffc-9add-f2c5e891412c service neutron] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:22.254042 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043f2288-1322-4ffc-9add-f2c5e891412c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:22.270256 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043f2288-1322-4ffc-9add-f2c5e891412c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:22.272683 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043f2288-1322-4ffc-9add-f2c5e891412c service neutron] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:22.292686 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 960/1945] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:27:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:27:22.450653 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585a8485-a64a-4f35-88d4-582549c1504a None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:22.452958 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-585a8485-a64a-4f35-88d4-582549c1504a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:27:22.480544 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585a8485-a64a-4f35-88d4-582549c1504a None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:22.488272 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa9f2fe-639c-4595-9806-6cc4965325be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:22.489581 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa9f2fe-639c-4595-9806-6cc4965325be admin admin] REQUEST_METHOD: `DELETE` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:22.490052 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa9f2fe-639c-4595-9806-6cc4965325be admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:22.490604 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa9f2fe-639c-4595-9806-6cc4965325be admin admin] PATH_INFO: `/v3/projects/6159a8f79fc84643a62c1b12af19787a` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:22.493083 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa9f2fe-639c-4595-9806-6cc4965325be admin admin] RBAC: Authorizing `identity:delete_project(project_id=6159a8f79fc84643a62c1b12af19787a)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:22.497775 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa9f2fe-639c-4595-9806-6cc4965325be admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:22.644643 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-3aa9f2fe-639c-4595-9806-6cc4965325be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6159a8f79fc84643a62c1b12af19787a', 'request_id': 'req-3aa9f2fe-639c-4595-9806-6cc4965325be'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:27:22.757577 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-3aa9f2fe-639c-4595-9806-6cc4965325be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6159a8f79fc84643a62c1b12af19787a', 'request_id': 'req-3aa9f2fe-639c-4595-9806-6cc4965325be'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:27:22.773349 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-3aa9f2fe-639c-4595-9806-6cc4965325be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6159a8f79fc84643a62c1b12af19787a', 'request_id': 'req-3aa9f2fe-639c-4595-9806-6cc4965325be'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:27:22.796401 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-3aa9f2fe-639c-4595-9806-6cc4965325be admin admin] The token cache is being invalidate because project 6159a8f79fc84643a62c1b12af19787a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81731) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:27:22.797870 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-3aa9f2fe-639c-4595-9806-6cc4965325be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:27:22.798363 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-3aa9f2fe-639c-4595-9806-6cc4965325be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:27:22.801322 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 986/1946] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 24 17:27:22 2022] DELETE /identity/v3/projects/6159a8f79fc84643a62c1b12af19787a => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:27:22.810690 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb51f98-3fd5-4ad0-a076-35ae8d406b84 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:22.811557 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb51f98-3fd5-4ad0-a076-35ae8d406b84 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:22.811713 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb51f98-3fd5-4ad0-a076-35ae8d406b84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:22.846630 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-9eb51f98-3fd5-4ad0-a076-35ae8d406b84 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:27:22.848448 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-9eb51f98-3fd5-4ad0-a076-35ae8d406b84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:27:22.878990 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:22.936675 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 961/1947] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:27:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:27:22.946058 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0ee7f9-0da6-4f01-8faf-a45f27069a8b None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:22.946608 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0ee7f9-0da6-4f01-8faf-a45f27069a8b None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:22.947567 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0ee7f9-0da6-4f01-8faf-a45f27069a8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:22.976242 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-2f0ee7f9-0da6-4f01-8faf-a45f27069a8b None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:27:22.979310 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-2f0ee7f9-0da6-4f01-8faf-a45f27069a8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:27:22.989228 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 987/1948] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:27:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:27:22.996074 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb51f98-3fd5-4ad0-a076-35ae8d406b84 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:23.007372 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb51f98-3fd5-4ad0-a076-35ae8d406b84 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:23.013861 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f408c0-a4de-4c51-b500-635cbb61e5cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:23.014502 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f408c0-a4de-4c51-b500-635cbb61e5cc admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:23.014785 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f408c0-a4de-4c51-b500-635cbb61e5cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:23.015098 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f408c0-a4de-4c51-b500-635cbb61e5cc admin admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:23.015775 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f408c0-a4de-4c51-b500-635cbb61e5cc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:27:23.016086 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f408c0-a4de-4c51-b500-635cbb61e5cc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:23.018128 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f408c0-a4de-4c51-b500-635cbb61e5cc admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:23.033181 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 962/1949] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 24 17:27:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:23.040193 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0ee7f9-0da6-4f01-8faf-a45f27069a8b None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:23.047003 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f408c0-a4de-4c51-b500-635cbb61e5cc admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:23.050509 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-d4f408c0-a4de-4c51-b500-635cbb61e5cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:27:23.057924 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0ee7f9-0da6-4f01-8faf-a45f27069a8b None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:23.068232 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f555888f-11cb-4f20-9070-ce638c3f2941 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:23.069694 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f555888f-11cb-4f20-9070-ce638c3f2941 admin admin] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:23.070204 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f555888f-11cb-4f20-9070-ce638c3f2941 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:23.070689 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f555888f-11cb-4f20-9070-ce638c3f2941 admin admin] PATH_INFO: `/v3/projects` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:23.071854 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f555888f-11cb-4f20-9070-ce638c3f2941 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:23.076495 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f555888f-11cb-4f20-9070-ce638c3f2941 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:23.077629 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f408c0-a4de-4c51-b500-635cbb61e5cc admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:23.086858 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c949d1-64ed-44fc-88b1-c562b8a681d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:23.088329 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c949d1-64ed-44fc-88b1-c562b8a681d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:23.089025 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c949d1-64ed-44fc-88b1-c562b8a681d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:23.089639 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c949d1-64ed-44fc-88b1-c562b8a681d4 admin admin] PATH_INFO: `/v3/projects/fcb3189f96454148b6502dae69fbfdbf` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:23.091912 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c949d1-64ed-44fc-88b1-c562b8a681d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fcb3189f96454148b6502dae69fbfdbf)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:23.095059 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c949d1-64ed-44fc-88b1-c562b8a681d4 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:23.166474 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 988/1950] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 24 17:27:23 2022] POST /identity/v3/projects => generated 379 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:27:23.172194 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f555888f-11cb-4f20-9070-ce638c3f2941 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:23.201845 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:23.207900 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-b1c949d1-64ed-44fc-88b1-c562b8a681d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fcb3189f96454148b6502dae69fbfdbf', 'request_id': 'req-b1c949d1-64ed-44fc-88b1-c562b8a681d4'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:27:23.257132 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f555888f-11cb-4f20-9070-ce638c3f2941 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:23.262931 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-b1c949d1-64ed-44fc-88b1-c562b8a681d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fcb3189f96454148b6502dae69fbfdbf', 'request_id': 'req-b1c949d1-64ed-44fc-88b1-c562b8a681d4'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:27:23.279306 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-b1c949d1-64ed-44fc-88b1-c562b8a681d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fcb3189f96454148b6502dae69fbfdbf', 'request_id': 'req-b1c949d1-64ed-44fc-88b1-c562b8a681d4'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:27:23.300498 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-b1c949d1-64ed-44fc-88b1-c562b8a681d4 admin admin] The token cache is being invalidate because project fcb3189f96454148b6502dae69fbfdbf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81732) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:27:23.301013 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-b1c949d1-64ed-44fc-88b1-c562b8a681d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:27:23.302257 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-b1c949d1-64ed-44fc-88b1-c562b8a681d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:27:23.305183 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 963/1951] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 24 17:27:23 2022] DELETE /identity/v3/projects/fcb3189f96454148b6502dae69fbfdbf => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:27:23.312155 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c949d1-64ed-44fc-88b1-c562b8a681d4 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:23.314562 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-b1c949d1-64ed-44fc-88b1-c562b8a681d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:27:23.355602 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fd2d62-4f99-4400-9bc9-6448541577ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:23.359563 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fd2d62-4f99-4400-9bc9-6448541577ef admin admin] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:23.359563 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fd2d62-4f99-4400-9bc9-6448541577ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:23.359563 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fd2d62-4f99-4400-9bc9-6448541577ef admin admin] PATH_INFO: `/v3/users` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:23.359563 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fd2d62-4f99-4400-9bc9-6448541577ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:23.359563 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fd2d62-4f99-4400-9bc9-6448541577ef admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:23.427041 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c949d1-64ed-44fc-88b1-c562b8a681d4 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:23.433548 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423fd409-ac7b-456c-abd0-3f1e5764e914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:23.434398 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423fd409-ac7b-456c-abd0-3f1e5764e914 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:23.434843 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423fd409-ac7b-456c-abd0-3f1e5764e914 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:23.435270 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423fd409-ac7b-456c-abd0-3f1e5764e914 admin admin] PATH_INFO: `/v3/users/99d7fdad92544e38877fff5bfccc56ba` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:23.436906 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423fd409-ac7b-456c-abd0-3f1e5764e914 admin admin] RBAC: Authorizing `identity:delete_user(user_id=99d7fdad92544e38877fff5bfccc56ba)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:23.439302 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423fd409-ac7b-456c-abd0-3f1e5764e914 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:23.480039 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 989/1952] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 24 17:27:23 2022] POST /identity/v3/users => generated 290 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:27:23.483789 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fd2d62-4f99-4400-9bc9-6448541577ef admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:23.487001 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-58fd2d62-4f99-4400-9bc9-6448541577ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:27:23.510109 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fd2d62-4f99-4400-9bc9-6448541577ef admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:23.517799 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff830362-fcba-4bf4-9e35-474398878568 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:23.518716 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff830362-fcba-4bf4-9e35-474398878568 admin admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:23.518993 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff830362-fcba-4bf4-9e35-474398878568 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:23.519273 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff830362-fcba-4bf4-9e35-474398878568 admin admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:23.520000 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff830362-fcba-4bf4-9e35-474398878568 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:23.523022 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff830362-fcba-4bf4-9e35-474398878568 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:23.540334 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 990/1953] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:27:23 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:27:23.545237 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff830362-fcba-4bf4-9e35-474398878568 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:23.565900 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff830362-fcba-4bf4-9e35-474398878568 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:23.573818 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d1b6e6-3d44-4f21-8e9d-208d0001787a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:23.574950 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d1b6e6-3d44-4f21-8e9d-208d0001787a admin admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:23.575451 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d1b6e6-3d44-4f21-8e9d-208d0001787a admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:23.575976 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d1b6e6-3d44-4f21-8e9d-208d0001787a admin admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:23.577515 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d1b6e6-3d44-4f21-8e9d-208d0001787a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:23.579975 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d1b6e6-3d44-4f21-8e9d-208d0001787a admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:23.607441 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 991/1954] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:27:23 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:27:23.612864 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d1b6e6-3d44-4f21-8e9d-208d0001787a admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:23.626892 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d1b6e6-3d44-4f21-8e9d-208d0001787a admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:23.633035 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb906f3-af2d-4b76-a0d3-44aa12cd54f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:23.633878 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb906f3-af2d-4b76-a0d3-44aa12cd54f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:23.634285 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb906f3-af2d-4b76-a0d3-44aa12cd54f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:23.634816 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb906f3-af2d-4b76-a0d3-44aa12cd54f9 admin admin] PATH_INFO: `/v3/projects/036fc4bf065249e29d0e1e90d23bae19/users/47da700d7d0f4d5cb1ea984a82d643bf/roles/60dc7e16aee746d5a3e8aa27d7d8525c` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:23.655216 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb906f3-af2d-4b76-a0d3-44aa12cd54f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=036fc4bf065249e29d0e1e90d23bae19, user_id=47da700d7d0f4d5cb1ea984a82d643bf, role_id=60dc7e16aee746d5a3e8aa27d7d8525c)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:23.660053 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb906f3-af2d-4b76-a0d3-44aa12cd54f9 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:23.675454 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:23.688507 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-423fd409-ac7b-456c-abd0-3f1e5764e914 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '99d7fdad92544e38877fff5bfccc56ba'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:27:23.769958 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 992/1955] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 24 17:27:23 2022] PUT /identity/v3/projects/036fc4bf065249e29d0e1e90d23bae19/users/47da700d7d0f4d5cb1ea984a82d643bf/roles/60dc7e16aee746d5a3e8aa27d7d8525c => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:27:23.771706 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-423fd409-ac7b-456c-abd0-3f1e5764e914 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '99d7fdad92544e38877fff5bfccc56ba'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:27:23.773189 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-423fd409-ac7b-456c-abd0-3f1e5764e914 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '99d7fdad92544e38877fff5bfccc56ba'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:27:23.778226 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaca134-3726-45ab-a870-dd5ad611c3be None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:23.778743 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaca134-3726-45ab-a870-dd5ad611c3be None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:23.779290 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaca134-3726-45ab-a870-dd5ad611c3be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:23.785954 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-423fd409-ac7b-456c-abd0-3f1e5764e914 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '99d7fdad92544e38877fff5bfccc56ba'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:27:23.789394 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-423fd409-ac7b-456c-abd0-3f1e5764e914 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '99d7fdad92544e38877fff5bfccc56ba'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:27:23.809408 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-7aaca134-3726-45ab-a870-dd5ad611c3be None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:27:23.810940 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-7aaca134-3726-45ab-a870-dd5ad611c3be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:27:23.812378 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 964/1956] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 24 17:27:23 2022] DELETE /identity/v3/users/99d7fdad92544e38877fff5bfccc56ba => generated 0 bytes in 501 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:27:23.926387 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 993/1957] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:27:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:27:23.941757 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423fd409-ac7b-456c-abd0-3f1e5764e914 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:23.944166 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-423fd409-ac7b-456c-abd0-3f1e5764e914 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:27:24.024396 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423fd409-ac7b-456c-abd0-3f1e5764e914 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:24.028222 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134dfd8a-0adc-47cd-aec1-067806795405 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e9ed4d47aef4140bfb2d3bb798cc559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:24.028897 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134dfd8a-0adc-47cd-aec1-067806795405 service neutron] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:24.029189 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134dfd8a-0adc-47cd-aec1-067806795405 service neutron] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:24.029475 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134dfd8a-0adc-47cd-aec1-067806795405 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:24.041113 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134dfd8a-0adc-47cd-aec1-067806795405 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:24.042575 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134dfd8a-0adc-47cd-aec1-067806795405 service neutron] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:24.054097 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 965/1958] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:27:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:27:24.411026 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaca134-3726-45ab-a870-dd5ad611c3be None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:24.413688 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-7aaca134-3726-45ab-a870-dd5ad611c3be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:27:24.434207 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaca134-3726-45ab-a870-dd5ad611c3be None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:24.439499 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22f3116-f8b4-4693-b2f2-a5c3a2414cd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:24.440293 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22f3116-f8b4-4693-b2f2-a5c3a2414cd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:24.440578 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22f3116-f8b4-4693-b2f2-a5c3a2414cd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:24.440851 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22f3116-f8b4-4693-b2f2-a5c3a2414cd5 admin admin] PATH_INFO: `/v3/projects/71440c8b913747ec97420968d48b753a` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:24.442660 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22f3116-f8b4-4693-b2f2-a5c3a2414cd5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=71440c8b913747ec97420968d48b753a)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:24.444466 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22f3116-f8b4-4693-b2f2-a5c3a2414cd5 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:24.448390 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:24.597132 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-a22f3116-f8b4-4693-b2f2-a5c3a2414cd5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71440c8b913747ec97420968d48b753a', 'request_id': 'req-a22f3116-f8b4-4693-b2f2-a5c3a2414cd5'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:27:24.673850 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-a22f3116-f8b4-4693-b2f2-a5c3a2414cd5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71440c8b913747ec97420968d48b753a', 'request_id': 'req-a22f3116-f8b4-4693-b2f2-a5c3a2414cd5'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:27:24.690424 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-a22f3116-f8b4-4693-b2f2-a5c3a2414cd5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71440c8b913747ec97420968d48b753a', 'request_id': 'req-a22f3116-f8b4-4693-b2f2-a5c3a2414cd5'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:27:24.722293 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-a22f3116-f8b4-4693-b2f2-a5c3a2414cd5 admin admin] The token cache is being invalidate because project 71440c8b913747ec97420968d48b753a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81731) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:27:24.723176 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-a22f3116-f8b4-4693-b2f2-a5c3a2414cd5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:27:24.724904 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-a22f3116-f8b4-4693-b2f2-a5c3a2414cd5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:27:24.735053 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 994/1959] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 24 17:27:24 2022] DELETE /identity/v3/projects/71440c8b913747ec97420968d48b753a => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:27:24.740559 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134dfd8a-0adc-47cd-aec1-067806795405 service neutron] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:24.762598 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134dfd8a-0adc-47cd-aec1-067806795405 service neutron] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:24.860289 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fdd00a-297d-4148-adf5-723af1e4bb94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:24.861346 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fdd00a-297d-4148-adf5-723af1e4bb94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:24.861639 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fdd00a-297d-4148-adf5-723af1e4bb94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:24.862048 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fdd00a-297d-4148-adf5-723af1e4bb94 admin admin] PATH_INFO: `/v3/users/53a4e528b55e49dfba20fad2425575c2` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:24.864231 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fdd00a-297d-4148-adf5-723af1e4bb94 admin admin] RBAC: Authorizing `identity:delete_user(user_id=53a4e528b55e49dfba20fad2425575c2)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:24.866246 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fdd00a-297d-4148-adf5-723af1e4bb94 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:25.074206 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-53fdd00a-297d-4148-adf5-723af1e4bb94 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53a4e528b55e49dfba20fad2425575c2'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:27:25.151561 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-53fdd00a-297d-4148-adf5-723af1e4bb94 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53a4e528b55e49dfba20fad2425575c2'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:27:25.152763 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-53fdd00a-297d-4148-adf5-723af1e4bb94 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53a4e528b55e49dfba20fad2425575c2'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:27:25.157686 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:25.167896 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-53fdd00a-297d-4148-adf5-723af1e4bb94 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53a4e528b55e49dfba20fad2425575c2'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:27:25.169161 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-53fdd00a-297d-4148-adf5-723af1e4bb94 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53a4e528b55e49dfba20fad2425575c2'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:27:25.204904 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 966/1960] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 24 17:27:24 2022] DELETE /identity/v3/users/53a4e528b55e49dfba20fad2425575c2 => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:27:25.962280 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22f3116-f8b4-4693-b2f2-a5c3a2414cd5 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:25.964128 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-a22f3116-f8b4-4693-b2f2-a5c3a2414cd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:27:25.983843 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22f3116-f8b4-4693-b2f2-a5c3a2414cd5 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:26.185009 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f6c25e-2312-4684-b405-6df47cc5a27d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:26.185009 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f6c25e-2312-4684-b405-6df47cc5a27d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:26.185009 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f6c25e-2312-4684-b405-6df47cc5a27d admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:26.185009 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f6c25e-2312-4684-b405-6df47cc5a27d admin admin] PATH_INFO: `/v3/projects/a5862a12b0b74b8fb05b0d7edff67dcd` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:26.185889 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f6c25e-2312-4684-b405-6df47cc5a27d admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5862a12b0b74b8fb05b0d7edff67dcd)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:26.185889 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f6c25e-2312-4684-b405-6df47cc5a27d admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:26.472212 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-c7f6c25e-2312-4684-b405-6df47cc5a27d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5862a12b0b74b8fb05b0d7edff67dcd', 'request_id': 'req-c7f6c25e-2312-4684-b405-6df47cc5a27d'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:27:26.557710 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-c7f6c25e-2312-4684-b405-6df47cc5a27d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5862a12b0b74b8fb05b0d7edff67dcd', 'request_id': 'req-c7f6c25e-2312-4684-b405-6df47cc5a27d'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:27:26.573501 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-c7f6c25e-2312-4684-b405-6df47cc5a27d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5862a12b0b74b8fb05b0d7edff67dcd', 'request_id': 'req-c7f6c25e-2312-4684-b405-6df47cc5a27d'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:27:26.618618 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-c7f6c25e-2312-4684-b405-6df47cc5a27d admin admin] The token cache is being invalidate because project a5862a12b0b74b8fb05b0d7edff67dcd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81731) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:27:26.620610 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-c7f6c25e-2312-4684-b405-6df47cc5a27d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:27:26.629901 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-c7f6c25e-2312-4684-b405-6df47cc5a27d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:27:26.644595 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 995/1961] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 24 17:27:25 2022] DELETE /identity/v3/projects/a5862a12b0b74b8fb05b0d7edff67dcd => generated 0 bytes in 684 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:27:26.659920 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2359e90d-f346-4ca6-8e74-81520c563f7d None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:26.660956 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2359e90d-f346-4ca6-8e74-81520c563f7d None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:26.661440 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2359e90d-f346-4ca6-8e74-81520c563f7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:26.686997 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-2359e90d-f346-4ca6-8e74-81520c563f7d None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:27:26.688252 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-2359e90d-f346-4ca6-8e74-81520c563f7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:27:26.794235 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 967/1962] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:27:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:27:26.808477 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f2f6d0-174b-451c-8037-90ab95b36e51 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:26.809115 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f2f6d0-174b-451c-8037-90ab95b36e51 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:26.809629 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f2f6d0-174b-451c-8037-90ab95b36e51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:26.843048 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-59f2f6d0-174b-451c-8037-90ab95b36e51 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:27:26.844996 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-59f2f6d0-174b-451c-8037-90ab95b36e51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:27:26.861064 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 996/1963] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:27:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:27:26.867118 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2359e90d-f346-4ca6-8e74-81520c563f7d None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:26.881172 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2359e90d-f346-4ca6-8e74-81520c563f7d None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:26.889204 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b508359-7c26-4eab-88d1-4a1077633642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:26.890199 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b508359-7c26-4eab-88d1-4a1077633642 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:26.890663 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b508359-7c26-4eab-88d1-4a1077633642 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:26.891093 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b508359-7c26-4eab-88d1-4a1077633642 admin admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:26.892296 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b508359-7c26-4eab-88d1-4a1077633642 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:27:26.892739 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b508359-7c26-4eab-88d1-4a1077633642 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:26.895186 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b508359-7c26-4eab-88d1-4a1077633642 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:26.916522 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 968/1964] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 24 17:27:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:26.923739 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f2f6d0-174b-451c-8037-90ab95b36e51 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:26.943211 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f2f6d0-174b-451c-8037-90ab95b36e51 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:26.967578 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e409b9-e13a-41cf-8f85-eb94bbbcb085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:26.968629 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e409b9-e13a-41cf-8f85-eb94bbbcb085 admin admin] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:26.968967 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e409b9-e13a-41cf-8f85-eb94bbbcb085 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:26.969369 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e409b9-e13a-41cf-8f85-eb94bbbcb085 admin admin] PATH_INFO: `/v3/projects` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:26.971862 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e409b9-e13a-41cf-8f85-eb94bbbcb085 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:26.975687 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e409b9-e13a-41cf-8f85-eb94bbbcb085 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:27.014542 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:27.092299 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 997/1965] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 24 17:27:26 2022] POST /identity/v3/projects => generated 371 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:27:27.097535 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b508359-7c26-4eab-88d1-4a1077633642 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:27.115098 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b508359-7c26-4eab-88d1-4a1077633642 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:27.201166 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d316d7a7-af2e-42f9-96fe-88544adddc8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:27.202342 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d316d7a7-af2e-42f9-96fe-88544adddc8b admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:27.202892 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d316d7a7-af2e-42f9-96fe-88544adddc8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:27.203452 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d316d7a7-af2e-42f9-96fe-88544adddc8b admin admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:27.204581 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d316d7a7-af2e-42f9-96fe-88544adddc8b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:27.208451 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d316d7a7-af2e-42f9-96fe-88544adddc8b admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:27.340027 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 969/1966] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 24 17:27:27 2022] POST /identity/v3/users => generated 286 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:27:27.345970 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e409b9-e13a-41cf-8f85-eb94bbbcb085 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:27.370470 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e409b9-e13a-41cf-8f85-eb94bbbcb085 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:27.384231 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9997f959-6af1-4b0d-9643-96607bb37372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:27.389822 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9997f959-6af1-4b0d-9643-96607bb37372 admin admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:27.390385 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9997f959-6af1-4b0d-9643-96607bb37372 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:27.390818 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9997f959-6af1-4b0d-9643-96607bb37372 admin admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:27.393115 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9997f959-6af1-4b0d-9643-96607bb37372 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:27.401545 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9997f959-6af1-4b0d-9643-96607bb37372 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:27.441549 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 998/1967] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:27:27 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:27:27.448767 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d316d7a7-af2e-42f9-96fe-88544adddc8b admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:27.466062 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d316d7a7-af2e-42f9-96fe-88544adddc8b admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:27.475125 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ea5e06-9586-4927-8129-74a129584fb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:27.476287 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ea5e06-9586-4927-8129-74a129584fb7 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:27.476855 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ea5e06-9586-4927-8129-74a129584fb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:27.477382 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ea5e06-9586-4927-8129-74a129584fb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:27.478640 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ea5e06-9586-4927-8129-74a129584fb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:27.482739 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ea5e06-9586-4927-8129-74a129584fb7 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:27.566622 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 970/1968] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:27:27 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:27:27.568074 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9997f959-6af1-4b0d-9643-96607bb37372 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:27.568074 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9997f959-6af1-4b0d-9643-96607bb37372 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:27.568074 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c654060c-6904-4fc5-9b9c-67063d2dee9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:27.568074 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c654060c-6904-4fc5-9b9c-67063d2dee9f admin admin] REQUEST_METHOD: `PUT` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:27.568699 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c654060c-6904-4fc5-9b9c-67063d2dee9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:27.568699 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c654060c-6904-4fc5-9b9c-67063d2dee9f admin admin] PATH_INFO: `/v3/projects/cc0fed6b580c40509ff6514dcd82f4dd/users/64eaec18a9d94723b541497bb5e1c105/roles/60dc7e16aee746d5a3e8aa27d7d8525c` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:27.568699 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c654060c-6904-4fc5-9b9c-67063d2dee9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc0fed6b580c40509ff6514dcd82f4dd, user_id=64eaec18a9d94723b541497bb5e1c105, role_id=60dc7e16aee746d5a3e8aa27d7d8525c)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:27.568699 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c654060c-6904-4fc5-9b9c-67063d2dee9f admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:27.832145 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 999/1969] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 24 17:27:27 2022] PUT /identity/v3/projects/cc0fed6b580c40509ff6514dcd82f4dd/users/64eaec18a9d94723b541497bb5e1c105/roles/60dc7e16aee746d5a3e8aa27d7d8525c => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:27:27.837725 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e9e036-e47e-4b0a-9085-d0e3a3dfbbb9 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:27.838270 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e9e036-e47e-4b0a-9085-d0e3a3dfbbb9 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:27.838270 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e9e036-e47e-4b0a-9085-d0e3a3dfbbb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:27.860726 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-06e9e036-e47e-4b0a-9085-d0e3a3dfbbb9 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:27:27.862549 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-06e9e036-e47e-4b0a-9085-d0e3a3dfbbb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:27:27.933204 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:27.984484 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 971/1970] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:27:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:27:27.998394 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c654060c-6904-4fc5-9b9c-67063d2dee9f admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:28.002064 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-c654060c-6904-4fc5-9b9c-67063d2dee9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:27:28.086148 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c654060c-6904-4fc5-9b9c-67063d2dee9f admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:28.089946 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d4e0da-bb1b-4c0b-9975-da889b76a23c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e9ed4d47aef4140bfb2d3bb798cc559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:28.090589 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d4e0da-bb1b-4c0b-9975-da889b76a23c service neutron] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:28.091031 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d4e0da-bb1b-4c0b-9975-da889b76a23c service neutron] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:28.091315 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d4e0da-bb1b-4c0b-9975-da889b76a23c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:28.107377 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d4e0da-bb1b-4c0b-9975-da889b76a23c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:28.109384 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d4e0da-bb1b-4c0b-9975-da889b76a23c service neutron] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:28.127233 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1000/1971] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:27:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:27:30.027117 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b31b93-1752-4ef8-aa5d-fa4473d939ac None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:30.028934 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b31b93-1752-4ef8-aa5d-fa4473d939ac None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:30.030045 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b31b93-1752-4ef8-aa5d-fa4473d939ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:30.082636 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-d4b31b93-1752-4ef8-aa5d-fa4473d939ac None None] MFA Rules not processed for user `64eaec18a9d94723b541497bb5e1c105`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:27:30.084373 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-d4b31b93-1752-4ef8-aa5d-fa4473d939ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:27:30.180008 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 972/1972] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:27:30 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:27:30.193376 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d4e0da-bb1b-4c0b-9975-da889b76a23c service neutron] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:30.195927 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-25d4e0da-bb1b-4c0b-9975-da889b76a23c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:27:30.287149 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d4e0da-bb1b-4c0b-9975-da889b76a23c service neutron] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:30.294061 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd6f836-41c8-4a08-becc-b04f586887bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be06246d47584b529150ceb98a5cfd95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:30.296676 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd6f836-41c8-4a08-becc-b04f586887bd service cinder] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:30.297272 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd6f836-41c8-4a08-becc-b04f586887bd service cinder] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:30.297703 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd6f836-41c8-4a08-becc-b04f586887bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:30.324530 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd6f836-41c8-4a08-becc-b04f586887bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:30.324530 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd6f836-41c8-4a08-becc-b04f586887bd service cinder] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:30.345745 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1001/1973] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:27:30 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:27:32.496572 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2472255c-265e-4998-ab7b-071ffe4cdc5e None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:32.497157 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2472255c-265e-4998-ab7b-071ffe4cdc5e None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:32.497270 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2472255c-265e-4998-ab7b-071ffe4cdc5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:32.531051 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-2472255c-265e-4998-ab7b-071ffe4cdc5e None None] MFA Rules not processed for user `47da700d7d0f4d5cb1ea984a82d643bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:27:32.532353 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-2472255c-265e-4998-ab7b-071ffe4cdc5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:27:32.630312 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 973/1974] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:27:32 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:27:32.635989 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd6f836-41c8-4a08-becc-b04f586887bd service cinder] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:32.638009 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-acd6f836-41c8-4a08-becc-b04f586887bd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:27:32.655802 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd6f836-41c8-4a08-becc-b04f586887bd service cinder] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:32.660800 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d39b44-6226-4780-9475-880356ab9b41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:32.661509 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d39b44-6226-4780-9475-880356ab9b41 admin admin] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:32.661791 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d39b44-6226-4780-9475-880356ab9b41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:32.662068 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d39b44-6226-4780-9475-880356ab9b41 admin admin] PATH_INFO: `/v3/projects` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:32.662771 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d39b44-6226-4780-9475-880356ab9b41 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:32.664956 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d39b44-6226-4780-9475-880356ab9b41 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:32.818098 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1002/1975] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 24 17:27:32 2022] POST /identity/v3/projects => generated 377 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:27:32.823119 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2472255c-265e-4998-ab7b-071ffe4cdc5e None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:32.834369 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2472255c-265e-4998-ab7b-071ffe4cdc5e None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:32.898096 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43e02de-d6ec-417d-971b-322a726a203b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:32.898713 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43e02de-d6ec-417d-971b-322a726a203b admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:32.899028 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43e02de-d6ec-417d-971b-322a726a203b admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:32.899303 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43e02de-d6ec-417d-971b-322a726a203b admin admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:32.900517 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43e02de-d6ec-417d-971b-322a726a203b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:32.902469 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43e02de-d6ec-417d-971b-322a726a203b admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:32.911838 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:33.079840 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 974/1976] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 24 17:27:32 2022] POST /identity/v3/users => generated 295 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:27:33.085279 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d39b44-6226-4780-9475-880356ab9b41 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:33.103001 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d39b44-6226-4780-9475-880356ab9b41 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:33.114459 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e94536-a9cf-40f9-a42c-9d2728498da8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:33.115540 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e94536-a9cf-40f9-a42c-9d2728498da8 admin admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:33.116026 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e94536-a9cf-40f9-a42c-9d2728498da8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:33.120788 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e94536-a9cf-40f9-a42c-9d2728498da8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:33.120788 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e94536-a9cf-40f9-a42c-9d2728498da8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:33.120788 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e94536-a9cf-40f9-a42c-9d2728498da8 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:33.140112 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1003/1977] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:27:33 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:27:33.146399 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43e02de-d6ec-417d-971b-322a726a203b admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:33.161397 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43e02de-d6ec-417d-971b-322a726a203b admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:33.168485 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9582f7-9a8f-4091-b953-00e1813e6fd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:33.169637 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9582f7-9a8f-4091-b953-00e1813e6fd1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:33.170136 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9582f7-9a8f-4091-b953-00e1813e6fd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:33.170607 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9582f7-9a8f-4091-b953-00e1813e6fd1 admin admin] PATH_INFO: `/v3/projects/1120639c2b904bdbbafa3bf6d6343270/users/6a9c759c8eeb46e4b1ef2c1c1e2a7201/roles/c5346bc54abf445f9900edf94264c850` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:33.206295 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9582f7-9a8f-4091-b953-00e1813e6fd1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1120639c2b904bdbbafa3bf6d6343270, user_id=6a9c759c8eeb46e4b1ef2c1c1e2a7201, role_id=c5346bc54abf445f9900edf94264c850)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:33.209076 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9582f7-9a8f-4091-b953-00e1813e6fd1 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:33.306708 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 975/1978] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 24 17:27:33 2022] PUT /identity/v3/projects/1120639c2b904bdbbafa3bf6d6343270/users/6a9c759c8eeb46e4b1ef2c1c1e2a7201/roles/c5346bc54abf445f9900edf94264c850 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:27:33.308958 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e94536-a9cf-40f9-a42c-9d2728498da8 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:33.322406 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e94536-a9cf-40f9-a42c-9d2728498da8 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:33.355268 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:33.389826 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11166a26-9bb3-446d-994b-8f160292d3d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:33.390599 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11166a26-9bb3-446d-994b-8f160292d3d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:33.391153 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11166a26-9bb3-446d-994b-8f160292d3d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:33.391825 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11166a26-9bb3-446d-994b-8f160292d3d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:33.392943 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11166a26-9bb3-446d-994b-8f160292d3d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:33.396144 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11166a26-9bb3-446d-994b-8f160292d3d1 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:33.413708 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1004/1979] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:27:33 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:27:33.418216 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9582f7-9a8f-4091-b953-00e1813e6fd1 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:33.432227 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9582f7-9a8f-4091-b953-00e1813e6fd1 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:33.439777 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7125eb4-d0df-4457-9e7f-3fdc27ded624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:33.440581 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7125eb4-d0df-4457-9e7f-3fdc27ded624 admin admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:33.440953 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7125eb4-d0df-4457-9e7f-3fdc27ded624 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:33.441331 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7125eb4-d0df-4457-9e7f-3fdc27ded624 admin admin] PATH_INFO: `/v3/projects/1120639c2b904bdbbafa3bf6d6343270/users/6a9c759c8eeb46e4b1ef2c1c1e2a7201/roles/c5346bc54abf445f9900edf94264c850` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:33.444069 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7125eb4-d0df-4457-9e7f-3fdc27ded624 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1120639c2b904bdbbafa3bf6d6343270, user_id=6a9c759c8eeb46e4b1ef2c1c1e2a7201, role_id=c5346bc54abf445f9900edf94264c850)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:33.447706 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7125eb4-d0df-4457-9e7f-3fdc27ded624 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:33.498358 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 976/1980] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 24 17:27:33 2022] PUT /identity/v3/projects/1120639c2b904bdbbafa3bf6d6343270/users/6a9c759c8eeb46e4b1ef2c1c1e2a7201/roles/c5346bc54abf445f9900edf94264c850 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:27:34.649354 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150b9840-e5ce-48eb-91ca-86ef6f00ec8e None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:34.650538 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150b9840-e5ce-48eb-91ca-86ef6f00ec8e None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:34.653410 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150b9840-e5ce-48eb-91ca-86ef6f00ec8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:34.695594 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-150b9840-e5ce-48eb-91ca-86ef6f00ec8e None None] MFA Rules not processed for user `64eaec18a9d94723b541497bb5e1c105`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:27:34.697386 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-150b9840-e5ce-48eb-91ca-86ef6f00ec8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:27:34.812448 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1005/1981] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:27:34 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:27:40.619848 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7125eb4-d0df-4457-9e7f-3fdc27ded624 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:40.621655 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-e7125eb4-d0df-4457-9e7f-3fdc27ded624 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:27:40.765320 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7125eb4-d0df-4457-9e7f-3fdc27ded624 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:40.783545 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e76c4d1-67e3-4e6b-9e3a-ed661a7b911a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:40.783545 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e76c4d1-67e3-4e6b-9e3a-ed661a7b911a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:40.783545 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e76c4d1-67e3-4e6b-9e3a-ed661a7b911a admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:40.783545 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e76c4d1-67e3-4e6b-9e3a-ed661a7b911a admin admin] PATH_INFO: `/v3/users/a1a0b876df274ea08e8c95c9b7f5a959` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:40.794382 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e76c4d1-67e3-4e6b-9e3a-ed661a7b911a admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1a0b876df274ea08e8c95c9b7f5a959)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:40.796423 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e76c4d1-67e3-4e6b-9e3a-ed661a7b911a admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:41.324865 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-6e76c4d1-67e3-4e6b-9e3a-ed661a7b911a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1a0b876df274ea08e8c95c9b7f5a959'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:27:41.425333 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-6e76c4d1-67e3-4e6b-9e3a-ed661a7b911a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1a0b876df274ea08e8c95c9b7f5a959'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:27:41.426159 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-6e76c4d1-67e3-4e6b-9e3a-ed661a7b911a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1a0b876df274ea08e8c95c9b7f5a959'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:27:41.475938 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-6e76c4d1-67e3-4e6b-9e3a-ed661a7b911a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1a0b876df274ea08e8c95c9b7f5a959'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:27:41.477207 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-6e76c4d1-67e3-4e6b-9e3a-ed661a7b911a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1a0b876df274ea08e8c95c9b7f5a959'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:27:41.540100 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 977/1982] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 24 17:27:40 2022] DELETE /identity/v3/users/a1a0b876df274ea08e8c95c9b7f5a959 => generated 0 bytes in 916 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:27:41.541978 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac0cfe7-76a1-4a20-8124-fc612ef79670 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:41.542996 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac0cfe7-76a1-4a20-8124-fc612ef79670 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:41.543275 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac0cfe7-76a1-4a20-8124-fc612ef79670 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:41.610629 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-9ac0cfe7-76a1-4a20-8124-fc612ef79670 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:27:41.612296 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-9ac0cfe7-76a1-4a20-8124-fc612ef79670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:27:41.834152 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1006/1983] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:27:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:27:41.852337 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e76c4d1-67e3-4e6b-9e3a-ed661a7b911a admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:41.857492 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-6e76c4d1-67e3-4e6b-9e3a-ed661a7b911a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:27:41.954889 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:INSERT=6,keystone:SELECT=84,keystone:DELETE=10 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:42.020953 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e76c4d1-67e3-4e6b-9e3a-ed661a7b911a admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:42.033604 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814edc57-0e08-4920-ba47-7b79ca97cf5c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e9ed4d47aef4140bfb2d3bb798cc559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:42.034601 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814edc57-0e08-4920-ba47-7b79ca97cf5c service neutron] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:42.035492 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814edc57-0e08-4920-ba47-7b79ca97cf5c service neutron] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:42.037635 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814edc57-0e08-4920-ba47-7b79ca97cf5c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:42.054489 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814edc57-0e08-4920-ba47-7b79ca97cf5c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:42.057415 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814edc57-0e08-4920-ba47-7b79ca97cf5c service neutron] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:42.089220 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 978/1984] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:27:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:27:43.062091 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac0cfe7-76a1-4a20-8124-fc612ef79670 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:43.062091 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-9ac0cfe7-76a1-4a20-8124-fc612ef79670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:27:43.070316 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac0cfe7-76a1-4a20-8124-fc612ef79670 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:43.085266 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390d7129-25ec-489b-bf9d-f84fb2538f4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:43.086139 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390d7129-25ec-489b-bf9d-f84fb2538f4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:43.086429 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390d7129-25ec-489b-bf9d-f84fb2538f4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:43.086710 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390d7129-25ec-489b-bf9d-f84fb2538f4f admin admin] PATH_INFO: `/v3/projects/e030c844d6544b4cbb9a9f8339d2c72c` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:43.092372 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390d7129-25ec-489b-bf9d-f84fb2538f4f admin admin] RBAC: Authorizing `identity:delete_project(project_id=e030c844d6544b4cbb9a9f8339d2c72c)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:43.093886 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390d7129-25ec-489b-bf9d-f84fb2538f4f admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:43.312854 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-390d7129-25ec-489b-bf9d-f84fb2538f4f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e030c844d6544b4cbb9a9f8339d2c72c', 'request_id': 'req-390d7129-25ec-489b-bf9d-f84fb2538f4f'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:27:43.404718 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-390d7129-25ec-489b-bf9d-f84fb2538f4f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e030c844d6544b4cbb9a9f8339d2c72c', 'request_id': 'req-390d7129-25ec-489b-bf9d-f84fb2538f4f'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:27:43.432155 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-390d7129-25ec-489b-bf9d-f84fb2538f4f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e030c844d6544b4cbb9a9f8339d2c72c', 'request_id': 'req-390d7129-25ec-489b-bf9d-f84fb2538f4f'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:27:43.445514 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:43.481456 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-390d7129-25ec-489b-bf9d-f84fb2538f4f admin admin] The token cache is being invalidate because project e030c844d6544b4cbb9a9f8339d2c72c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81731) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:27:43.482470 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-390d7129-25ec-489b-bf9d-f84fb2538f4f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:27:43.484001 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-390d7129-25ec-489b-bf9d-f84fb2538f4f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:27:43.487443 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1007/1985] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 24 17:27:43 2022] DELETE /identity/v3/projects/e030c844d6544b4cbb9a9f8339d2c72c => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:27:43.496160 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f8ee2b-a0e1-48b9-8c56-4f978922140d None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:43.497052 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f8ee2b-a0e1-48b9-8c56-4f978922140d None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:43.497052 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f8ee2b-a0e1-48b9-8c56-4f978922140d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:43.540527 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-65f8ee2b-a0e1-48b9-8c56-4f978922140d None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:27:43.542022 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-65f8ee2b-a0e1-48b9-8c56-4f978922140d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:27:43.662416 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 979/1986] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:27:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:27:43.674943 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a307b4f-6199-4c8a-b6c3-f7751f108570 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:43.675634 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a307b4f-6199-4c8a-b6c3-f7751f108570 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:43.676247 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a307b4f-6199-4c8a-b6c3-f7751f108570 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:43.710479 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-5a307b4f-6199-4c8a-b6c3-f7751f108570 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:27:43.712472 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-5a307b4f-6199-4c8a-b6c3-f7751f108570 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:27:43.731694 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1008/1987] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:27:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:27:43.737294 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f8ee2b-a0e1-48b9-8c56-4f978922140d None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:43.757947 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f8ee2b-a0e1-48b9-8c56-4f978922140d None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:43.767572 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c00e9e9-6667-4885-803f-b66f0bcab710 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:43.768947 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c00e9e9-6667-4885-803f-b66f0bcab710 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:43.769695 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c00e9e9-6667-4885-803f-b66f0bcab710 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:43.770345 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c00e9e9-6667-4885-803f-b66f0bcab710 admin admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:43.771753 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c00e9e9-6667-4885-803f-b66f0bcab710 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:27:43.772346 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c00e9e9-6667-4885-803f-b66f0bcab710 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:43.774321 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c00e9e9-6667-4885-803f-b66f0bcab710 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:43.805227 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 980/1988] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 24 17:27:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:43.809040 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a307b4f-6199-4c8a-b6c3-f7751f108570 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:43.827884 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a307b4f-6199-4c8a-b6c3-f7751f108570 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:43.838297 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b633442e-134d-40c8-8f06-79236bea26b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:43.839183 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b633442e-134d-40c8-8f06-79236bea26b8 admin admin] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:43.839602 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b633442e-134d-40c8-8f06-79236bea26b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:43.839921 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b633442e-134d-40c8-8f06-79236bea26b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:43.840915 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b633442e-134d-40c8-8f06-79236bea26b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:43.843143 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b633442e-134d-40c8-8f06-79236bea26b8 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:44.024418 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1009/1989] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 24 17:27:43 2022] POST /identity/v3/projects => generated 373 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:27:44.027151 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c00e9e9-6667-4885-803f-b66f0bcab710 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:44.049359 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c00e9e9-6667-4885-803f-b66f0bcab710 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:44.052403 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c525f260-321c-4ec3-9e17-098ebb366ef6 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:44.052926 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c525f260-321c-4ec3-9e17-098ebb366ef6 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:44.054268 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c525f260-321c-4ec3-9e17-098ebb366ef6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:44.111481 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-c525f260-321c-4ec3-9e17-098ebb366ef6 None None] MFA Rules not processed for user `6a9c759c8eeb46e4b1ef2c1c1e2a7201`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:27:44.112709 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-c525f260-321c-4ec3-9e17-098ebb366ef6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:27:44.180583 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be61e5ab-f222-4872-9236-fd56ba034f34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:44.181874 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be61e5ab-f222-4872-9236-fd56ba034f34 admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:44.182434 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be61e5ab-f222-4872-9236-fd56ba034f34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:44.182797 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be61e5ab-f222-4872-9236-fd56ba034f34 admin admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:44.184086 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be61e5ab-f222-4872-9236-fd56ba034f34 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:44.185911 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be61e5ab-f222-4872-9236-fd56ba034f34 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:44.260889 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1010/1990] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:27:44 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:27:44.286661 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c525f260-321c-4ec3-9e17-098ebb366ef6 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:44.289649 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-c525f260-321c-4ec3-9e17-098ebb366ef6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:27:44.366799 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 981/1991] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 24 17:27:44 2022] POST /identity/v3/users => generated 287 bytes in 341 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:27:44.372011 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be61e5ab-f222-4872-9236-fd56ba034f34 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:44.399079 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be61e5ab-f222-4872-9236-fd56ba034f34 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:44.426044 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd749e3-d0a7-4ba8-bf72-bd9aa17e7efa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:44.426044 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd749e3-d0a7-4ba8-bf72-bd9aa17e7efa admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:44.426044 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd749e3-d0a7-4ba8-bf72-bd9aa17e7efa admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:44.426476 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c525f260-321c-4ec3-9e17-098ebb366ef6 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:44.426626 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd749e3-d0a7-4ba8-bf72-bd9aa17e7efa admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:44.426626 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd749e3-d0a7-4ba8-bf72-bd9aa17e7efa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:44.426626 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd749e3-d0a7-4ba8-bf72-bd9aa17e7efa admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:44.427072 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff2af94-3382-4b3d-9a3f-2f0c0a8a24f3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557aca9255cb4bbb8af52e796c5c5177', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:44.427072 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff2af94-3382-4b3d-9a3f-2f0c0a8a24f3 service nova] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:44.427072 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff2af94-3382-4b3d-9a3f-2f0c0a8a24f3 service nova] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:44.427072 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff2af94-3382-4b3d-9a3f-2f0c0a8a24f3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:44.427602 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-1ff2af94-3382-4b3d-9a3f-2f0c0a8a24f3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:27:44.456060 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 982/1992] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:27:44 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:27:44.461323 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd749e3-d0a7-4ba8-bf72-bd9aa17e7efa admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:44.485611 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd749e3-d0a7-4ba8-bf72-bd9aa17e7efa admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:44.495332 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56370e30-2489-4f82-9fc4-4883c82327e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:44.499827 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56370e30-2489-4f82-9fc4-4883c82327e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:44.499827 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56370e30-2489-4f82-9fc4-4883c82327e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:44.499827 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56370e30-2489-4f82-9fc4-4883c82327e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:44.499827 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56370e30-2489-4f82-9fc4-4883c82327e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:44.501029 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56370e30-2489-4f82-9fc4-4883c82327e0 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:44.534280 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 983/1993] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:27:44 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:27:44.538689 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56370e30-2489-4f82-9fc4-4883c82327e0 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:44.558141 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:44.566831 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56370e30-2489-4f82-9fc4-4883c82327e0 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:44.570218 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff2af94-3382-4b3d-9a3f-2f0c0a8a24f3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:44.572619 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff2af94-3382-4b3d-9a3f-2f0c0a8a24f3 service nova] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:44.577339 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99015816-0a2c-4b32-b1af-a1caa0633ebd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:44.578872 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99015816-0a2c-4b32-b1af-a1caa0633ebd admin admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:44.579189 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99015816-0a2c-4b32-b1af-a1caa0633ebd admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:44.579529 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99015816-0a2c-4b32-b1af-a1caa0633ebd admin admin] PATH_INFO: `/v3/projects/854fbe174dd645449c0746ad88e62885/users/3b3e4ef9731046069f19d08ec97670a1/roles/60dc7e16aee746d5a3e8aa27d7d8525c` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:44.594396 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1011/1994] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:27:44 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:27:44.664266 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99015816-0a2c-4b32-b1af-a1caa0633ebd admin admin] RBAC: Authorizing `identity:create_grant(project_id=854fbe174dd645449c0746ad88e62885, user_id=3b3e4ef9731046069f19d08ec97670a1, role_id=60dc7e16aee746d5a3e8aa27d7d8525c)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:44.677259 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99015816-0a2c-4b32-b1af-a1caa0633ebd admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:44.828174 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 984/1995] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 24 17:27:44 2022] PUT /identity/v3/projects/854fbe174dd645449c0746ad88e62885/users/3b3e4ef9731046069f19d08ec97670a1/roles/60dc7e16aee746d5a3e8aa27d7d8525c => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:27:44.841308 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c01fb9f-d9af-4715-af38-3deb8335c62b None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:44.841699 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c01fb9f-d9af-4715-af38-3deb8335c62b None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:44.842076 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c01fb9f-d9af-4715-af38-3deb8335c62b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:44.948962 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-3c01fb9f-d9af-4715-af38-3deb8335c62b None None] MFA Rules not processed for user `3b3e4ef9731046069f19d08ec97670a1`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:27:44.951071 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-3c01fb9f-d9af-4715-af38-3deb8335c62b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:27:45.147232 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1012/1996] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:27:44 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 308 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:27:45.177914 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99015816-0a2c-4b32-b1af-a1caa0633ebd admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:45.179887 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-99015816-0a2c-4b32-b1af-a1caa0633ebd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:27:45.200083 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:45.332033 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99015816-0a2c-4b32-b1af-a1caa0633ebd admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:45.337547 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95bf70e-5333-4a82-a3d2-0925ec0e27ea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be06246d47584b529150ceb98a5cfd95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:45.338354 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95bf70e-5333-4a82-a3d2-0925ec0e27ea service cinder] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:45.339076 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95bf70e-5333-4a82-a3d2-0925ec0e27ea service cinder] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:45.339752 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95bf70e-5333-4a82-a3d2-0925ec0e27ea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:45.351900 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95bf70e-5333-4a82-a3d2-0925ec0e27ea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:45.354074 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95bf70e-5333-4a82-a3d2-0925ec0e27ea service cinder] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:45.366753 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 985/1997] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:27:45 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:27:46.082573 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c01fb9f-d9af-4715-af38-3deb8335c62b None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:46.084576 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-3c01fb9f-d9af-4715-af38-3deb8335c62b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:27:46.170424 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c01fb9f-d9af-4715-af38-3deb8335c62b None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:46.181580 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb91398-a0cf-4ad7-adf3-188163d1f757 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:46.181580 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb91398-a0cf-4ad7-adf3-188163d1f757 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:46.181580 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb91398-a0cf-4ad7-adf3-188163d1f757 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:46.181580 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb91398-a0cf-4ad7-adf3-188163d1f757 admin admin] PATH_INFO: `/v3/users/bb2bc3ba87a549018d50fe3c9c6720d0` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:46.182290 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb91398-a0cf-4ad7-adf3-188163d1f757 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb2bc3ba87a549018d50fe3c9c6720d0)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:46.182290 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb91398-a0cf-4ad7-adf3-188163d1f757 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:46.476357 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-ffb91398-a0cf-4ad7-adf3-188163d1f757 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb2bc3ba87a549018d50fe3c9c6720d0'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:27:46.489106 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-ffb91398-a0cf-4ad7-adf3-188163d1f757 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb2bc3ba87a549018d50fe3c9c6720d0'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:27:46.615437 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-ffb91398-a0cf-4ad7-adf3-188163d1f757 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb2bc3ba87a549018d50fe3c9c6720d0'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:27:46.620306 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-ffb91398-a0cf-4ad7-adf3-188163d1f757 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb2bc3ba87a549018d50fe3c9c6720d0'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:27:46.631918 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-ffb91398-a0cf-4ad7-adf3-188163d1f757 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb2bc3ba87a549018d50fe3c9c6720d0'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:27:46.634704 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:46.642992 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1013/1998] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 24 17:27:46 2022] DELETE /identity/v3/users/bb2bc3ba87a549018d50fe3c9c6720d0 => generated 0 bytes in 561 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:27:47.083837 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95bf70e-5333-4a82-a3d2-0925ec0e27ea service cinder] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:47.094008 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95bf70e-5333-4a82-a3d2-0925ec0e27ea service cinder] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:47.214815 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06af256-f70e-4724-9940-e61a990a981f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:47.215530 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06af256-f70e-4724-9940-e61a990a981f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:47.215815 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06af256-f70e-4724-9940-e61a990a981f admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:47.223059 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06af256-f70e-4724-9940-e61a990a981f admin admin] PATH_INFO: `/v3/projects/879c5673d2cf49f5973d1faa5f774c43` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:47.229011 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06af256-f70e-4724-9940-e61a990a981f admin admin] RBAC: Authorizing `identity:delete_project(project_id=879c5673d2cf49f5973d1faa5f774c43)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:47.229011 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06af256-f70e-4724-9940-e61a990a981f admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:47.388118 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-d06af256-f70e-4724-9940-e61a990a981f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '879c5673d2cf49f5973d1faa5f774c43', 'request_id': 'req-d06af256-f70e-4724-9940-e61a990a981f'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:27:47.476563 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-d06af256-f70e-4724-9940-e61a990a981f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '879c5673d2cf49f5973d1faa5f774c43', 'request_id': 'req-d06af256-f70e-4724-9940-e61a990a981f'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:27:47.487840 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-d06af256-f70e-4724-9940-e61a990a981f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '879c5673d2cf49f5973d1faa5f774c43', 'request_id': 'req-d06af256-f70e-4724-9940-e61a990a981f'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:27:47.503399 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-d06af256-f70e-4724-9940-e61a990a981f admin admin] The token cache is being invalidate because project 879c5673d2cf49f5973d1faa5f774c43 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81732) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:27:47.503399 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-d06af256-f70e-4724-9940-e61a990a981f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:27:47.504234 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-d06af256-f70e-4724-9940-e61a990a981f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:27:47.505835 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 986/1999] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 24 17:27:47 2022] DELETE /identity/v3/projects/879c5673d2cf49f5973d1faa5f774c43 => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:27:47.513264 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95932dae-cc23-49e7-b5ff-9daad1c5bfaa None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:47.513989 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95932dae-cc23-49e7-b5ff-9daad1c5bfaa None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:47.514171 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95932dae-cc23-49e7-b5ff-9daad1c5bfaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:47.539922 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-95932dae-cc23-49e7-b5ff-9daad1c5bfaa None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:27:47.541707 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-95932dae-cc23-49e7-b5ff-9daad1c5bfaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:27:47.642165 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1014/2000] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:27:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:27:47.650009 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a571bac-f15c-4e1c-bc2d-11e83101b546 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:47.650301 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a571bac-f15c-4e1c-bc2d-11e83101b546 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:47.650584 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a571bac-f15c-4e1c-bc2d-11e83101b546 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:47.679491 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-6a571bac-f15c-4e1c-bc2d-11e83101b546 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:27:47.683060 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-6a571bac-f15c-4e1c-bc2d-11e83101b546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:27:47.690404 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 987/2001] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:27:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:27:47.695938 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95932dae-cc23-49e7-b5ff-9daad1c5bfaa None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:47.709675 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95932dae-cc23-49e7-b5ff-9daad1c5bfaa None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:47.714626 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904175dc-4f57-4c0a-a7d3-afe741b3b00d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:47.715296 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904175dc-4f57-4c0a-a7d3-afe741b3b00d admin admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:47.715615 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904175dc-4f57-4c0a-a7d3-afe741b3b00d admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:47.716028 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904175dc-4f57-4c0a-a7d3-afe741b3b00d admin admin] PATH_INFO: `/v3/domains` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:47.716829 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904175dc-4f57-4c0a-a7d3-afe741b3b00d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:27:47.717007 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904175dc-4f57-4c0a-a7d3-afe741b3b00d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:47.718734 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904175dc-4f57-4c0a-a7d3-afe741b3b00d admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:47.733055 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1015/2002] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 24 17:27:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:27:47.737805 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a571bac-f15c-4e1c-bc2d-11e83101b546 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:47.757594 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a571bac-f15c-4e1c-bc2d-11e83101b546 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:47.757594 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e8250a-5dc9-4127-aebf-bd0808ad7de3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:47.757594 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e8250a-5dc9-4127-aebf-bd0808ad7de3 admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:47.757594 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e8250a-5dc9-4127-aebf-bd0808ad7de3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:47.757937 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e8250a-5dc9-4127-aebf-bd0808ad7de3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:47.757937 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e8250a-5dc9-4127-aebf-bd0808ad7de3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:47.757937 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e8250a-5dc9-4127-aebf-bd0808ad7de3 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:47.902017 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 988/2003] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 24 17:27:47 2022] POST /identity/v3/projects => generated 405 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:27:47.904298 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904175dc-4f57-4c0a-a7d3-afe741b3b00d admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:47.916626 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904175dc-4f57-4c0a-a7d3-afe741b3b00d admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:47.985815 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f54a31-9999-482e-9dbb-d63f607bd9bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:47.987483 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f54a31-9999-482e-9dbb-d63f607bd9bf admin admin] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:47.987483 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f54a31-9999-482e-9dbb-d63f607bd9bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:47.987483 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f54a31-9999-482e-9dbb-d63f607bd9bf admin admin] PATH_INFO: `/v3/users` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:47.987919 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f54a31-9999-482e-9dbb-d63f607bd9bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:47.989605 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f54a31-9999-482e-9dbb-d63f607bd9bf admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:48.122713 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1016/2004] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 24 17:27:47 2022] POST /identity/v3/users => generated 303 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:27:48.128208 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e8250a-5dc9-4127-aebf-bd0808ad7de3 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:48.146516 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e8250a-5dc9-4127-aebf-bd0808ad7de3 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:48.156688 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38282eb3-52b4-418e-ad64-d32da584a392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:48.158175 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38282eb3-52b4-418e-ad64-d32da584a392 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:48.158411 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38282eb3-52b4-418e-ad64-d32da584a392 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:48.158686 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38282eb3-52b4-418e-ad64-d32da584a392 admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:48.159406 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38282eb3-52b4-418e-ad64-d32da584a392 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:48.161185 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38282eb3-52b4-418e-ad64-d32da584a392 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:48.164539 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:48.175483 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 989/2005] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:27:48 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:27:48.180099 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f54a31-9999-482e-9dbb-d63f607bd9bf admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:48.196964 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f54a31-9999-482e-9dbb-d63f607bd9bf admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:48.204463 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699243e3-b3ee-410d-a657-5cedf35702c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:48.205462 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699243e3-b3ee-410d-a657-5cedf35702c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:48.205841 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699243e3-b3ee-410d-a657-5cedf35702c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:48.206174 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699243e3-b3ee-410d-a657-5cedf35702c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:48.206930 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699243e3-b3ee-410d-a657-5cedf35702c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:48.209608 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699243e3-b3ee-410d-a657-5cedf35702c9 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:48.224022 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1017/2006] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:27:48 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:27:48.233993 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38282eb3-52b4-418e-ad64-d32da584a392 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:48.247847 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38282eb3-52b4-418e-ad64-d32da584a392 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:48.256272 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfb185e-491e-4b62-bce7-ec5c26c24213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:48.257124 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfb185e-491e-4b62-bce7-ec5c26c24213 admin admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:48.257526 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfb185e-491e-4b62-bce7-ec5c26c24213 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:48.257903 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfb185e-491e-4b62-bce7-ec5c26c24213 admin admin] PATH_INFO: `/v3/projects/ec39472524fe4f2c8f4d5d14f2e5b804/users/2beb575721e847c6b2e8b1706a71e7dd/roles/60dc7e16aee746d5a3e8aa27d7d8525c` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:48.280762 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfb185e-491e-4b62-bce7-ec5c26c24213 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec39472524fe4f2c8f4d5d14f2e5b804, user_id=2beb575721e847c6b2e8b1706a71e7dd, role_id=60dc7e16aee746d5a3e8aa27d7d8525c)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:48.283882 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfb185e-491e-4b62-bce7-ec5c26c24213 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:48.385403 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 990/2007] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 24 17:27:48 2022] PUT /identity/v3/projects/ec39472524fe4f2c8f4d5d14f2e5b804/users/2beb575721e847c6b2e8b1706a71e7dd/roles/60dc7e16aee746d5a3e8aa27d7d8525c => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:27:48.401744 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b2eed8-72aa-46a3-b45e-eba5763cf389 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:48.402043 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b2eed8-72aa-46a3-b45e-eba5763cf389 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:48.402317 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b2eed8-72aa-46a3-b45e-eba5763cf389 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:48.432727 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-75b2eed8-72aa-46a3-b45e-eba5763cf389 None None] MFA Rules not processed for user `2beb575721e847c6b2e8b1706a71e7dd`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:27:48.433969 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-75b2eed8-72aa-46a3-b45e-eba5763cf389 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:27:48.514098 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1018/2008] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:27:48 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:27:48.518996 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfb185e-491e-4b62-bce7-ec5c26c24213 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:48.594245 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfb185e-491e-4b62-bce7-ec5c26c24213 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:48.599252 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a5001d-3949-4aa2-891d-3fef57784e93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:48.600309 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a5001d-3949-4aa2-891d-3fef57784e93 admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:48.600664 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a5001d-3949-4aa2-891d-3fef57784e93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:48.600942 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a5001d-3949-4aa2-891d-3fef57784e93 admin admin] PATH_INFO: `/v3/projects` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:48.601573 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a5001d-3949-4aa2-891d-3fef57784e93 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:48.603578 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a5001d-3949-4aa2-891d-3fef57784e93 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:48.781853 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 991/2009] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 24 17:27:48 2022] POST /identity/v3/projects => generated 405 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:27:48.786090 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b2eed8-72aa-46a3-b45e-eba5763cf389 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:48.797880 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b2eed8-72aa-46a3-b45e-eba5763cf389 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:48.886540 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec2e46f-8b46-442c-8dad-7fa3626a4cf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:48.890074 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec2e46f-8b46-442c-8dad-7fa3626a4cf3 admin admin] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:48.890074 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec2e46f-8b46-442c-8dad-7fa3626a4cf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:48.890074 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec2e46f-8b46-442c-8dad-7fa3626a4cf3 admin admin] PATH_INFO: `/v3/users` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:48.890074 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec2e46f-8b46-442c-8dad-7fa3626a4cf3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:48.891065 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec2e46f-8b46-442c-8dad-7fa3626a4cf3 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:49.041957 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1019/2010] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 24 17:27:48 2022] POST /identity/v3/users => generated 309 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:27:49.044859 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a5001d-3949-4aa2-891d-3fef57784e93 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:49.057716 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a5001d-3949-4aa2-891d-3fef57784e93 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:49.079772 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d142c2-6f8a-49dd-8b39-0d708b720ed0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:49.080455 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d142c2-6f8a-49dd-8b39-0d708b720ed0 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:49.080738 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d142c2-6f8a-49dd-8b39-0d708b720ed0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:49.081010 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d142c2-6f8a-49dd-8b39-0d708b720ed0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:49.081659 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d142c2-6f8a-49dd-8b39-0d708b720ed0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:49.083364 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d142c2-6f8a-49dd-8b39-0d708b720ed0 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:49.126499 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 992/2011] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:27:49 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:27:49.130505 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec2e46f-8b46-442c-8dad-7fa3626a4cf3 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:49.145808 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec2e46f-8b46-442c-8dad-7fa3626a4cf3 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:49.153359 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab1521c-7081-40e2-abe0-e6b58c4bd26d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:49.154047 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab1521c-7081-40e2-abe0-e6b58c4bd26d admin admin] REQUEST_METHOD: `PUT` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:49.154326 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab1521c-7081-40e2-abe0-e6b58c4bd26d admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:49.154848 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab1521c-7081-40e2-abe0-e6b58c4bd26d admin admin] PATH_INFO: `/v3/projects/86566a51693048c0bd93da9ccacf6385/users/d76255ed6a5649938b5f54990b8ceaa2/roles/c5346bc54abf445f9900edf94264c850` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:49.171598 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab1521c-7081-40e2-abe0-e6b58c4bd26d admin admin] RBAC: Authorizing `identity:create_grant(project_id=86566a51693048c0bd93da9ccacf6385, user_id=d76255ed6a5649938b5f54990b8ceaa2, role_id=c5346bc54abf445f9900edf94264c850)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:49.174323 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab1521c-7081-40e2-abe0-e6b58c4bd26d admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:49.319880 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d142c2-6f8a-49dd-8b39-0d708b720ed0 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:49.330293 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1020/2012] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 24 17:27:49 2022] PUT /identity/v3/projects/86566a51693048c0bd93da9ccacf6385/users/d76255ed6a5649938b5f54990b8ceaa2/roles/c5346bc54abf445f9900edf94264c850 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:27:49.335416 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d142c2-6f8a-49dd-8b39-0d708b720ed0 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:49.429599 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac13240-b335-40f8-b0dd-95b876748794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:49.430429 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac13240-b335-40f8-b0dd-95b876748794 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:49.431403 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac13240-b335-40f8-b0dd-95b876748794 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:49.431692 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac13240-b335-40f8-b0dd-95b876748794 admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:49.432411 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac13240-b335-40f8-b0dd-95b876748794 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:49.434486 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac13240-b335-40f8-b0dd-95b876748794 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:49.445764 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 993/2013] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:27:49 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:27:49.450638 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab1521c-7081-40e2-abe0-e6b58c4bd26d admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:49.461433 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab1521c-7081-40e2-abe0-e6b58c4bd26d admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:49.466430 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3549e8f-f6ce-4b25-a7fd-dbe83f24e006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:49.467122 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3549e8f-f6ce-4b25-a7fd-dbe83f24e006 admin admin] REQUEST_METHOD: `PUT` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:49.467407 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3549e8f-f6ce-4b25-a7fd-dbe83f24e006 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:49.467682 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3549e8f-f6ce-4b25-a7fd-dbe83f24e006 admin admin] PATH_INFO: `/v3/projects/86566a51693048c0bd93da9ccacf6385/users/d76255ed6a5649938b5f54990b8ceaa2/roles/c5346bc54abf445f9900edf94264c850` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:49.469587 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3549e8f-f6ce-4b25-a7fd-dbe83f24e006 admin admin] RBAC: Authorizing `identity:create_grant(project_id=86566a51693048c0bd93da9ccacf6385, user_id=d76255ed6a5649938b5f54990b8ceaa2, role_id=c5346bc54abf445f9900edf94264c850)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:49.472254 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3549e8f-f6ce-4b25-a7fd-dbe83f24e006 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:49.483506 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1021/2014] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 24 17:27:49 2022] PUT /identity/v3/projects/86566a51693048c0bd93da9ccacf6385/users/d76255ed6a5649938b5f54990b8ceaa2/roles/c5346bc54abf445f9900edf94264c850 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:27:49.499756 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6172ba-2ef7-4496-80a7-eaf1e8f3fb19 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:49.500116 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6172ba-2ef7-4496-80a7-eaf1e8f3fb19 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:49.500391 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6172ba-2ef7-4496-80a7-eaf1e8f3fb19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:49.540852 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-8a6172ba-2ef7-4496-80a7-eaf1e8f3fb19 None None] MFA Rules not processed for user `d76255ed6a5649938b5f54990b8ceaa2`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:27:49.542789 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-8a6172ba-2ef7-4496-80a7-eaf1e8f3fb19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:27:49.658285 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 994/2015] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:27:49 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:27:49.922913 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:50.212279 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3549e8f-f6ce-4b25-a7fd-dbe83f24e006 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:50.215101 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-e3549e8f-f6ce-4b25-a7fd-dbe83f24e006 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:27:50.321421 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3549e8f-f6ce-4b25-a7fd-dbe83f24e006 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:50.329225 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ac51aa-84e5-4806-89a7-6bd22ad0ab82 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be06246d47584b529150ceb98a5cfd95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:50.330503 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ac51aa-84e5-4806-89a7-6bd22ad0ab82 service cinder] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:50.330793 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ac51aa-84e5-4806-89a7-6bd22ad0ab82 service cinder] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:50.331105 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ac51aa-84e5-4806-89a7-6bd22ad0ab82 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:50.365518 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ac51aa-84e5-4806-89a7-6bd22ad0ab82 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:50.367881 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ac51aa-84e5-4806-89a7-6bd22ad0ab82 service cinder] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:50.406017 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1022/2016] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:27:50 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:27:50.682293 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6172ba-2ef7-4496-80a7-eaf1e8f3fb19 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:50.753034 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6172ba-2ef7-4496-80a7-eaf1e8f3fb19 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:50.757626 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6980a440-66cd-460c-9576-453812af4f84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:50.758267 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6980a440-66cd-460c-9576-453812af4f84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:50.758550 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6980a440-66cd-460c-9576-453812af4f84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:50.759385 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6980a440-66cd-460c-9576-453812af4f84 admin admin] PATH_INFO: `/v3/users/2beb575721e847c6b2e8b1706a71e7dd` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:50.760569 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6980a440-66cd-460c-9576-453812af4f84 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2beb575721e847c6b2e8b1706a71e7dd)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:50.762723 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6980a440-66cd-460c-9576-453812af4f84 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:51.000378 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-6980a440-66cd-460c-9576-453812af4f84 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2beb575721e847c6b2e8b1706a71e7dd'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:27:51.064417 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-6980a440-66cd-460c-9576-453812af4f84 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2beb575721e847c6b2e8b1706a71e7dd'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:27:51.065054 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-6980a440-66cd-460c-9576-453812af4f84 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2beb575721e847c6b2e8b1706a71e7dd'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:27:51.074110 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-6980a440-66cd-460c-9576-453812af4f84 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2beb575721e847c6b2e8b1706a71e7dd'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:27:51.074850 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-6980a440-66cd-460c-9576-453812af4f84 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2beb575721e847c6b2e8b1706a71e7dd'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:27:51.115096 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 995/2017] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 24 17:27:50 2022] DELETE /identity/v3/users/2beb575721e847c6b2e8b1706a71e7dd => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:27:51.120313 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b4368f-2f6c-4c67-8fde-314ee8a4578f None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:51.121087 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b4368f-2f6c-4c67-8fde-314ee8a4578f None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:51.121512 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b4368f-2f6c-4c67-8fde-314ee8a4578f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:51.156653 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-22b4368f-2f6c-4c67-8fde-314ee8a4578f None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:27:51.158896 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-22b4368f-2f6c-4c67-8fde-314ee8a4578f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:27:51.245997 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1023/2018] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:27:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:27:51.256053 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6980a440-66cd-460c-9576-453812af4f84 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:51.257524 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-6980a440-66cd-460c-9576-453812af4f84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:27:51.330315 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6980a440-66cd-460c-9576-453812af4f84 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:51.334473 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c972d3df-5bd3-457b-ae7d-8a21073cbac7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e9ed4d47aef4140bfb2d3bb798cc559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:51.337673 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c972d3df-5bd3-457b-ae7d-8a21073cbac7 service neutron] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:51.337673 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c972d3df-5bd3-457b-ae7d-8a21073cbac7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:51.337673 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c972d3df-5bd3-457b-ae7d-8a21073cbac7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:51.351743 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c972d3df-5bd3-457b-ae7d-8a21073cbac7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:51.353356 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c972d3df-5bd3-457b-ae7d-8a21073cbac7 service neutron] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:51.364458 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 996/2019] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:27:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:27:52.166579 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b4368f-2f6c-4c67-8fde-314ee8a4578f None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:52.168624 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-22b4368f-2f6c-4c67-8fde-314ee8a4578f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:27:52.192583 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b4368f-2f6c-4c67-8fde-314ee8a4578f None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:52.199201 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba77922-6bd6-471e-a7d5-e38c941cf1a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:52.200225 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba77922-6bd6-471e-a7d5-e38c941cf1a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:52.200503 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba77922-6bd6-471e-a7d5-e38c941cf1a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:52.200777 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba77922-6bd6-471e-a7d5-e38c941cf1a6 admin admin] PATH_INFO: `/v3/projects/ec39472524fe4f2c8f4d5d14f2e5b804` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:52.202081 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba77922-6bd6-471e-a7d5-e38c941cf1a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec39472524fe4f2c8f4d5d14f2e5b804)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:52.204024 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba77922-6bd6-471e-a7d5-e38c941cf1a6 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:52.391088 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-aba77922-6bd6-471e-a7d5-e38c941cf1a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec39472524fe4f2c8f4d5d14f2e5b804', 'request_id': 'req-aba77922-6bd6-471e-a7d5-e38c941cf1a6'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:27:52.501264 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-aba77922-6bd6-471e-a7d5-e38c941cf1a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec39472524fe4f2c8f4d5d14f2e5b804', 'request_id': 'req-aba77922-6bd6-471e-a7d5-e38c941cf1a6'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:27:52.521814 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-aba77922-6bd6-471e-a7d5-e38c941cf1a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec39472524fe4f2c8f4d5d14f2e5b804', 'request_id': 'req-aba77922-6bd6-471e-a7d5-e38c941cf1a6'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:27:52.547624 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-aba77922-6bd6-471e-a7d5-e38c941cf1a6 admin admin] The token cache is being invalidate because project ec39472524fe4f2c8f4d5d14f2e5b804 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81731) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:27:52.548349 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-aba77922-6bd6-471e-a7d5-e38c941cf1a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:27:52.556045 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-aba77922-6bd6-471e-a7d5-e38c941cf1a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:27:52.563640 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1024/2020] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 24 17:27:52 2022] DELETE /identity/v3/projects/ec39472524fe4f2c8f4d5d14f2e5b804 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:27:52.568990 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c972d3df-5bd3-457b-ae7d-8a21073cbac7 service neutron] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:52.593173 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c972d3df-5bd3-457b-ae7d-8a21073cbac7 service neutron] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:52.598065 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:52.725596 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:52.736111 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad646984-b21a-4029-8d58-d9e024ba12e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:52.740460 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad646984-b21a-4029-8d58-d9e024ba12e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:52.740460 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad646984-b21a-4029-8d58-d9e024ba12e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:52.740460 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad646984-b21a-4029-8d58-d9e024ba12e3 admin admin] PATH_INFO: `/v3/users/d76255ed6a5649938b5f54990b8ceaa2` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:52.740460 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad646984-b21a-4029-8d58-d9e024ba12e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d76255ed6a5649938b5f54990b8ceaa2)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:52.741960 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad646984-b21a-4029-8d58-d9e024ba12e3 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:53.082973 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-ad646984-b21a-4029-8d58-d9e024ba12e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd76255ed6a5649938b5f54990b8ceaa2'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:27:53.185510 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-ad646984-b21a-4029-8d58-d9e024ba12e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd76255ed6a5649938b5f54990b8ceaa2'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:27:53.186142 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-ad646984-b21a-4029-8d58-d9e024ba12e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd76255ed6a5649938b5f54990b8ceaa2'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:27:53.194111 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-ad646984-b21a-4029-8d58-d9e024ba12e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd76255ed6a5649938b5f54990b8ceaa2'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:27:53.194967 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-ad646984-b21a-4029-8d58-d9e024ba12e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd76255ed6a5649938b5f54990b8ceaa2'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:27:53.219175 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 997/2021] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 24 17:27:52 2022] DELETE /identity/v3/users/d76255ed6a5649938b5f54990b8ceaa2 => generated 0 bytes in 651 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:27:53.973728 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba77922-6bd6-471e-a7d5-e38c941cf1a6 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:53.975990 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-aba77922-6bd6-471e-a7d5-e38c941cf1a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:27:53.992462 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba77922-6bd6-471e-a7d5-e38c941cf1a6 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:54.066029 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44365403-2d7c-4e6a-8e01-3c3c03607279 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:54.067054 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44365403-2d7c-4e6a-8e01-3c3c03607279 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:54.067720 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44365403-2d7c-4e6a-8e01-3c3c03607279 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:54.068024 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44365403-2d7c-4e6a-8e01-3c3c03607279 admin admin] PATH_INFO: `/v3/projects/86566a51693048c0bd93da9ccacf6385` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:54.069317 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44365403-2d7c-4e6a-8e01-3c3c03607279 admin admin] RBAC: Authorizing `identity:delete_project(project_id=86566a51693048c0bd93da9ccacf6385)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:54.072078 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44365403-2d7c-4e6a-8e01-3c3c03607279 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:54.194556 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-44365403-2d7c-4e6a-8e01-3c3c03607279 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '86566a51693048c0bd93da9ccacf6385', 'request_id': 'req-44365403-2d7c-4e6a-8e01-3c3c03607279'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:27:54.302536 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-44365403-2d7c-4e6a-8e01-3c3c03607279 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '86566a51693048c0bd93da9ccacf6385', 'request_id': 'req-44365403-2d7c-4e6a-8e01-3c3c03607279'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:27:54.313680 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-44365403-2d7c-4e6a-8e01-3c3c03607279 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '86566a51693048c0bd93da9ccacf6385', 'request_id': 'req-44365403-2d7c-4e6a-8e01-3c3c03607279'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:27:54.328814 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-44365403-2d7c-4e6a-8e01-3c3c03607279 admin admin] The token cache is being invalidate because project 86566a51693048c0bd93da9ccacf6385 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81731) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:27:54.329337 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-44365403-2d7c-4e6a-8e01-3c3c03607279 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:27:54.330416 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-44365403-2d7c-4e6a-8e01-3c3c03607279 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:27:54.332233 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1025/2022] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 24 17:27:53 2022] DELETE /identity/v3/projects/86566a51693048c0bd93da9ccacf6385 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:27:54.339831 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b1a807-51a4-41b8-93b6-d85ca39f31f2 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:54.340724 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b1a807-51a4-41b8-93b6-d85ca39f31f2 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:54.340724 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b1a807-51a4-41b8-93b6-d85ca39f31f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:54.363638 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-e2b1a807-51a4-41b8-93b6-d85ca39f31f2 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:27:54.365121 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-e2b1a807-51a4-41b8-93b6-d85ca39f31f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:27:54.456424 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 998/2023] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:27:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:27:54.467361 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86593304-ed2c-483d-b6f7-dc3b63e96e8f None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:54.468003 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86593304-ed2c-483d-b6f7-dc3b63e96e8f None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:54.468579 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86593304-ed2c-483d-b6f7-dc3b63e96e8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:54.491552 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-86593304-ed2c-483d-b6f7-dc3b63e96e8f None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:27:54.492802 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-86593304-ed2c-483d-b6f7-dc3b63e96e8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:27:54.501268 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1026/2024] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:27:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:27:54.506497 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b1a807-51a4-41b8-93b6-d85ca39f31f2 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:54.518298 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b1a807-51a4-41b8-93b6-d85ca39f31f2 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:54.523099 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483ee6f7-e1a0-4731-8c37-81bf58f4d689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:54.523735 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483ee6f7-e1a0-4731-8c37-81bf58f4d689 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:54.524056 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483ee6f7-e1a0-4731-8c37-81bf58f4d689 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:54.524863 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483ee6f7-e1a0-4731-8c37-81bf58f4d689 admin admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:54.525486 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483ee6f7-e1a0-4731-8c37-81bf58f4d689 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:27:54.525790 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483ee6f7-e1a0-4731-8c37-81bf58f4d689 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:54.527446 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483ee6f7-e1a0-4731-8c37-81bf58f4d689 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:54.541248 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 999/2025] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 24 17:27:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:27:54.546502 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86593304-ed2c-483d-b6f7-dc3b63e96e8f None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:54.564251 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86593304-ed2c-483d-b6f7-dc3b63e96e8f None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:54.578015 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3ebb54-9c3f-4fe3-aba2-10bd19b6fb73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:54.578206 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:54.578419 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3ebb54-9c3f-4fe3-aba2-10bd19b6fb73 admin admin] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:54.578419 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3ebb54-9c3f-4fe3-aba2-10bd19b6fb73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:54.578419 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3ebb54-9c3f-4fe3-aba2-10bd19b6fb73 admin admin] PATH_INFO: `/v3/projects` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:54.578419 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3ebb54-9c3f-4fe3-aba2-10bd19b6fb73 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:54.578419 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3ebb54-9c3f-4fe3-aba2-10bd19b6fb73 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:54.632174 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:54.717074 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1027/2026] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 24 17:27:54 2022] POST /identity/v3/projects => generated 381 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:27:54.718131 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483ee6f7-e1a0-4731-8c37-81bf58f4d689 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:54.746458 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483ee6f7-e1a0-4731-8c37-81bf58f4d689 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:54.816697 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaea1c59-efcb-412e-b500-0428fa858593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:54.817321 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaea1c59-efcb-412e-b500-0428fa858593 admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:54.817598 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaea1c59-efcb-412e-b500-0428fa858593 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:54.817869 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaea1c59-efcb-412e-b500-0428fa858593 admin admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:54.818569 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaea1c59-efcb-412e-b500-0428fa858593 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:54.821948 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaea1c59-efcb-412e-b500-0428fa858593 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:54.972717 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1000/2027] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 24 17:27:54 2022] POST /identity/v3/users => generated 291 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:27:54.979518 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3ebb54-9c3f-4fe3-aba2-10bd19b6fb73 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:54.999465 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3ebb54-9c3f-4fe3-aba2-10bd19b6fb73 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:55.010971 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de36b889-5f77-4356-9ec5-75f0cc125d3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:55.012467 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de36b889-5f77-4356-9ec5-75f0cc125d3f admin admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:55.012982 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de36b889-5f77-4356-9ec5-75f0cc125d3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:55.013485 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de36b889-5f77-4356-9ec5-75f0cc125d3f admin admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:55.014842 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de36b889-5f77-4356-9ec5-75f0cc125d3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:55.018409 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de36b889-5f77-4356-9ec5-75f0cc125d3f admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:55.045615 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1028/2028] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:27:54 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:27:55.052791 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaea1c59-efcb-412e-b500-0428fa858593 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:55.073163 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaea1c59-efcb-412e-b500-0428fa858593 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:55.081057 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbfb224-9921-4187-af42-15a479e356b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:55.082358 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbfb224-9921-4187-af42-15a479e356b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:55.082864 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbfb224-9921-4187-af42-15a479e356b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:55.083444 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbfb224-9921-4187-af42-15a479e356b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:55.084688 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbfb224-9921-4187-af42-15a479e356b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:55.087719 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbfb224-9921-4187-af42-15a479e356b4 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:55.111283 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1001/2029] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:27:55 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:27:55.116607 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de36b889-5f77-4356-9ec5-75f0cc125d3f admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:55.147840 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de36b889-5f77-4356-9ec5-75f0cc125d3f admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:55.159286 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c087c0-cdda-4c3a-b79b-07b28808531d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:55.160925 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c087c0-cdda-4c3a-b79b-07b28808531d admin admin] REQUEST_METHOD: `PUT` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:55.161641 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c087c0-cdda-4c3a-b79b-07b28808531d admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:55.162601 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c087c0-cdda-4c3a-b79b-07b28808531d admin admin] PATH_INFO: `/v3/projects/987b521e613c42f38771a63cc2345499/users/f5ec7641876b4cf5b5d3585758502746/roles/60dc7e16aee746d5a3e8aa27d7d8525c` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:55.191390 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c087c0-cdda-4c3a-b79b-07b28808531d admin admin] RBAC: Authorizing `identity:create_grant(project_id=987b521e613c42f38771a63cc2345499, user_id=f5ec7641876b4cf5b5d3585758502746, role_id=60dc7e16aee746d5a3e8aa27d7d8525c)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:55.194202 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c087c0-cdda-4c3a-b79b-07b28808531d admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:55.393517 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1029/2030] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 24 17:27:55 2022] PUT /identity/v3/projects/987b521e613c42f38771a63cc2345499/users/f5ec7641876b4cf5b5d3585758502746/roles/60dc7e16aee746d5a3e8aa27d7d8525c => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:27:55.407037 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fe465a-afe2-4e3c-af7b-2697c013b8b8 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:55.407921 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fe465a-afe2-4e3c-af7b-2697c013b8b8 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:55.407921 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fe465a-afe2-4e3c-af7b-2697c013b8b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:55.447241 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-44fe465a-afe2-4e3c-af7b-2697c013b8b8 None None] MFA Rules not processed for user `f5ec7641876b4cf5b5d3585758502746`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:27:55.448644 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-44fe465a-afe2-4e3c-af7b-2697c013b8b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:27:55.562280 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1002/2031] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:27:55 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:27:55.567036 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c087c0-cdda-4c3a-b79b-07b28808531d admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:55.694869 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c087c0-cdda-4c3a-b79b-07b28808531d admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:55.713326 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490d9929-a2c6-4dbb-84d9-c9888462a745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:55.713951 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490d9929-a2c6-4dbb-84d9-c9888462a745 admin admin] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:55.714659 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490d9929-a2c6-4dbb-84d9-c9888462a745 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:55.714659 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490d9929-a2c6-4dbb-84d9-c9888462a745 admin admin] PATH_INFO: `/v3/projects` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:55.715402 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490d9929-a2c6-4dbb-84d9-c9888462a745 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:55.718180 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490d9929-a2c6-4dbb-84d9-c9888462a745 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:55.857086 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1030/2032] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 24 17:27:55 2022] POST /identity/v3/projects => generated 375 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:27:55.864423 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fe465a-afe2-4e3c-af7b-2697c013b8b8 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:55.885902 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fe465a-afe2-4e3c-af7b-2697c013b8b8 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:55.996244 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4665d729-6a6e-4658-ae71-fcde1e111cc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:55.996885 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4665d729-6a6e-4658-ae71-fcde1e111cc8 admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:55.997173 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4665d729-6a6e-4658-ae71-fcde1e111cc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:55.997479 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4665d729-6a6e-4658-ae71-fcde1e111cc8 admin admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:55.998472 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4665d729-6a6e-4658-ae71-fcde1e111cc8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:56.003789 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4665d729-6a6e-4658-ae71-fcde1e111cc8 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:56.244132 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1003/2033] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 24 17:27:55 2022] POST /identity/v3/users => generated 294 bytes in 381 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:27:56.248051 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490d9929-a2c6-4dbb-84d9-c9888462a745 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:56.264102 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490d9929-a2c6-4dbb-84d9-c9888462a745 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:56.279549 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e7e09c-eb4d-491d-abd6-381a22033990 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:56.281058 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e7e09c-eb4d-491d-abd6-381a22033990 admin admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:56.281813 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e7e09c-eb4d-491d-abd6-381a22033990 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:56.282583 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e7e09c-eb4d-491d-abd6-381a22033990 admin admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:56.284134 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e7e09c-eb4d-491d-abd6-381a22033990 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:56.288176 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e7e09c-eb4d-491d-abd6-381a22033990 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:56.320716 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1031/2034] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:27:56 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:27:56.325733 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4665d729-6a6e-4658-ae71-fcde1e111cc8 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:56.338471 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4665d729-6a6e-4658-ae71-fcde1e111cc8 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:56.344451 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eefa55c-2ec8-4c08-b8ba-a6de83196805 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:56.345280 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eefa55c-2ec8-4c08-b8ba-a6de83196805 admin admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:56.345675 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eefa55c-2ec8-4c08-b8ba-a6de83196805 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:56.346215 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eefa55c-2ec8-4c08-b8ba-a6de83196805 admin admin] PATH_INFO: `/v3/projects/95d64b5d18d04a88a0723d2e3bf9a81d/users/809966567b8c464dbcf694da271b65da/roles/c5346bc54abf445f9900edf94264c850` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:56.390183 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eefa55c-2ec8-4c08-b8ba-a6de83196805 admin admin] RBAC: Authorizing `identity:create_grant(project_id=95d64b5d18d04a88a0723d2e3bf9a81d, user_id=809966567b8c464dbcf694da271b65da, role_id=c5346bc54abf445f9900edf94264c850)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:56.393269 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eefa55c-2ec8-4c08-b8ba-a6de83196805 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:56.643062 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1004/2035] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 24 17:27:56 2022] PUT /identity/v3/projects/95d64b5d18d04a88a0723d2e3bf9a81d/users/809966567b8c464dbcf694da271b65da/roles/c5346bc54abf445f9900edf94264c850 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:27:56.645676 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e7e09c-eb4d-491d-abd6-381a22033990 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:56.665764 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e7e09c-eb4d-491d-abd6-381a22033990 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:56.779750 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:56.828654 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb40451-b327-4347-93cc-7bef1192a93a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:56.829648 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb40451-b327-4347-93cc-7bef1192a93a admin admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:56.830109 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb40451-b327-4347-93cc-7bef1192a93a admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:56.830471 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb40451-b327-4347-93cc-7bef1192a93a admin admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:56.831629 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb40451-b327-4347-93cc-7bef1192a93a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:56.834690 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb40451-b327-4347-93cc-7bef1192a93a admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:56.867293 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eefa55c-2ec8-4c08-b8ba-a6de83196805 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:56.880399 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1032/2036] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:27:56 2022] GET /identity/v3/roles => generated 1420 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:27:56.880989 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eefa55c-2ec8-4c08-b8ba-a6de83196805 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:56.895744 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dafd4b7-0ad6-4912-8b32-8b13be8c6c7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:56.897250 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dafd4b7-0ad6-4912-8b32-8b13be8c6c7c admin admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:56.897612 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dafd4b7-0ad6-4912-8b32-8b13be8c6c7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:56.898101 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dafd4b7-0ad6-4912-8b32-8b13be8c6c7c admin admin] PATH_INFO: `/v3/projects/95d64b5d18d04a88a0723d2e3bf9a81d/users/809966567b8c464dbcf694da271b65da/roles/c5346bc54abf445f9900edf94264c850` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:56.902880 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dafd4b7-0ad6-4912-8b32-8b13be8c6c7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=95d64b5d18d04a88a0723d2e3bf9a81d, user_id=809966567b8c464dbcf694da271b65da, role_id=c5346bc54abf445f9900edf94264c850)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:56.908291 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dafd4b7-0ad6-4912-8b32-8b13be8c6c7c admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:56.943528 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1005/2037] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 24 17:27:56 2022] PUT /identity/v3/projects/95d64b5d18d04a88a0723d2e3bf9a81d/users/809966567b8c464dbcf694da271b65da/roles/c5346bc54abf445f9900edf94264c850 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:27:56.947684 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaeeac8-2637-4449-9889-c16e880dba2d None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:56.947959 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaeeac8-2637-4449-9889-c16e880dba2d None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:56.948474 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaeeac8-2637-4449-9889-c16e880dba2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:56.979024 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:27:57.009758 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-bbaeeac8-2637-4449-9889-c16e880dba2d None None] MFA Rules not processed for user `809966567b8c464dbcf694da271b65da`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:27:57.011142 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-bbaeeac8-2637-4449-9889-c16e880dba2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:27:57.144178 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1033/2038] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:27:56 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:27:57.221333 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dafd4b7-0ad6-4912-8b32-8b13be8c6c7c admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:57.223465 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-8dafd4b7-0ad6-4912-8b32-8b13be8c6c7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:27:57.313993 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dafd4b7-0ad6-4912-8b32-8b13be8c6c7c admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:57.318103 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd7347d-75a1-4f8c-b149-fd0a3b308419 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be06246d47584b529150ceb98a5cfd95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:57.318756 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd7347d-75a1-4f8c-b149-fd0a3b308419 service cinder] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:57.319084 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd7347d-75a1-4f8c-b149-fd0a3b308419 service cinder] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:57.319543 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd7347d-75a1-4f8c-b149-fd0a3b308419 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:57.331482 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd7347d-75a1-4f8c-b149-fd0a3b308419 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:57.334163 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd7347d-75a1-4f8c-b149-fd0a3b308419 service cinder] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:57.346427 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1006/2039] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:27:57 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:27:57.767300 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbaeeac8-2637-4449-9889-c16e880dba2d None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:27:57.769900 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-bbaeeac8-2637-4449-9889-c16e880dba2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:27:57.783472 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbaeeac8-2637-4449-9889-c16e880dba2d None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:27:57.788838 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232d82ae-688e-4ca4-bacf-3e0a871d481b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be06246d47584b529150ceb98a5cfd95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:27:57.789456 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232d82ae-688e-4ca4-bacf-3e0a871d481b service cinder] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:27:57.789795 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232d82ae-688e-4ca4-bacf-3e0a871d481b service cinder] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:27:57.790031 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232d82ae-688e-4ca4-bacf-3e0a871d481b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:27:57.879432 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232d82ae-688e-4ca4-bacf-3e0a871d481b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:27:57.881544 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232d82ae-688e-4ca4-bacf-3e0a871d481b service cinder] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:27:57.902645 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1034/2040] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:27:57 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:27:58.959576 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:05.530524 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd7347d-75a1-4f8c-b149-fd0a3b308419 service cinder] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:05.532113 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-3bd7347d-75a1-4f8c-b149-fd0a3b308419 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:28:05.632173 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd7347d-75a1-4f8c-b149-fd0a3b308419 service cinder] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:05.637591 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5658d0-7b69-4cf0-94ef-42c77df67472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:05.638265 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5658d0-7b69-4cf0-94ef-42c77df67472 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:05.638559 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5658d0-7b69-4cf0-94ef-42c77df67472 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:05.638833 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5658d0-7b69-4cf0-94ef-42c77df67472 admin admin] PATH_INFO: `/v3/users/3b3e4ef9731046069f19d08ec97670a1` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:05.640053 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5658d0-7b69-4cf0-94ef-42c77df67472 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b3e4ef9731046069f19d08ec97670a1)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:05.642318 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5658d0-7b69-4cf0-94ef-42c77df67472 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:05.864219 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-6b5658d0-7b69-4cf0-94ef-42c77df67472 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b3e4ef9731046069f19d08ec97670a1'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:28:05.944960 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-6b5658d0-7b69-4cf0-94ef-42c77df67472 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b3e4ef9731046069f19d08ec97670a1'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:28:05.945607 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-6b5658d0-7b69-4cf0-94ef-42c77df67472 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b3e4ef9731046069f19d08ec97670a1'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:28:05.954291 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-6b5658d0-7b69-4cf0-94ef-42c77df67472 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b3e4ef9731046069f19d08ec97670a1'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:28:05.955014 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-6b5658d0-7b69-4cf0-94ef-42c77df67472 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b3e4ef9731046069f19d08ec97670a1'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:28:05.958119 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:05.986454 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1007/2041] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 24 17:28:05 2022] DELETE /identity/v3/users/3b3e4ef9731046069f19d08ec97670a1 => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:28:05.991475 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190660cc-b3a9-469b-b0bb-9cee5ac09ea3 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:05.992352 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190660cc-b3a9-469b-b0bb-9cee5ac09ea3 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:05.992504 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190660cc-b3a9-469b-b0bb-9cee5ac09ea3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:06.015744 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-190660cc-b3a9-469b-b0bb-9cee5ac09ea3 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:28:06.017231 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-190660cc-b3a9-469b-b0bb-9cee5ac09ea3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:28:06.093550 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1035/2042] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:28:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:28:06.103783 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5658d0-7b69-4cf0-94ef-42c77df67472 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:06.105349 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-6b5658d0-7b69-4cf0-94ef-42c77df67472 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:28:06.177004 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5658d0-7b69-4cf0-94ef-42c77df67472 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:06.181294 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02ad040-c5cd-40a1-b1dc-fc50cc8af283 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e9ed4d47aef4140bfb2d3bb798cc559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:06.182031 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02ad040-c5cd-40a1-b1dc-fc50cc8af283 service neutron] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:06.182411 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02ad040-c5cd-40a1-b1dc-fc50cc8af283 service neutron] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:06.182861 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02ad040-c5cd-40a1-b1dc-fc50cc8af283 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:06.193392 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02ad040-c5cd-40a1-b1dc-fc50cc8af283 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:06.195018 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02ad040-c5cd-40a1-b1dc-fc50cc8af283 service neutron] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:06.206131 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1008/2043] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:28:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:28:06.816607 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190660cc-b3a9-469b-b0bb-9cee5ac09ea3 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:06.824893 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-190660cc-b3a9-469b-b0bb-9cee5ac09ea3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:28:06.833384 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190660cc-b3a9-469b-b0bb-9cee5ac09ea3 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:06.841114 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf23a0c-e18d-4f5a-a6d7-3ca6ec81ad3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:06.847735 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf23a0c-e18d-4f5a-a6d7-3ca6ec81ad3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:06.847735 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf23a0c-e18d-4f5a-a6d7-3ca6ec81ad3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:06.847735 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf23a0c-e18d-4f5a-a6d7-3ca6ec81ad3f admin admin] PATH_INFO: `/v3/projects/854fbe174dd645449c0746ad88e62885` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:06.847735 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf23a0c-e18d-4f5a-a6d7-3ca6ec81ad3f admin admin] RBAC: Authorizing `identity:delete_project(project_id=854fbe174dd645449c0746ad88e62885)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:06.847735 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf23a0c-e18d-4f5a-a6d7-3ca6ec81ad3f admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:06.987536 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:07.003502 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-5cf23a0c-e18d-4f5a-a6d7-3ca6ec81ad3f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '854fbe174dd645449c0746ad88e62885', 'request_id': 'req-5cf23a0c-e18d-4f5a-a6d7-3ca6ec81ad3f'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:28:07.098470 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-5cf23a0c-e18d-4f5a-a6d7-3ca6ec81ad3f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '854fbe174dd645449c0746ad88e62885', 'request_id': 'req-5cf23a0c-e18d-4f5a-a6d7-3ca6ec81ad3f'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:28:07.105959 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-5cf23a0c-e18d-4f5a-a6d7-3ca6ec81ad3f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '854fbe174dd645449c0746ad88e62885', 'request_id': 'req-5cf23a0c-e18d-4f5a-a6d7-3ca6ec81ad3f'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:28:07.116809 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-5cf23a0c-e18d-4f5a-a6d7-3ca6ec81ad3f admin admin] The token cache is being invalidate because project 854fbe174dd645449c0746ad88e62885 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81731) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:28:07.117093 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-5cf23a0c-e18d-4f5a-a6d7-3ca6ec81ad3f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:28:07.117736 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-5cf23a0c-e18d-4f5a-a6d7-3ca6ec81ad3f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:28:07.119241 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1036/2044] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 24 17:28:06 2022] DELETE /identity/v3/projects/854fbe174dd645449c0746ad88e62885 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:28:07.124156 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d74b926-63c8-439e-95a1-de2775a61e01 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:07.124692 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d74b926-63c8-439e-95a1-de2775a61e01 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:07.124692 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d74b926-63c8-439e-95a1-de2775a61e01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:07.145396 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-7d74b926-63c8-439e-95a1-de2775a61e01 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:28:07.146588 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-7d74b926-63c8-439e-95a1-de2775a61e01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:28:07.205952 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1009/2045] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:28:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:28:07.215252 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d534af-5da9-4f29-a7ce-a7e86d8f39b1 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:07.215545 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d534af-5da9-4f29-a7ce-a7e86d8f39b1 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:07.215818 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d534af-5da9-4f29-a7ce-a7e86d8f39b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:07.235113 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-76d534af-5da9-4f29-a7ce-a7e86d8f39b1 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:28:07.236319 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-76d534af-5da9-4f29-a7ce-a7e86d8f39b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:28:07.244413 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1037/2046] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:28:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:28:07.248599 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d74b926-63c8-439e-95a1-de2775a61e01 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:07.259420 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d74b926-63c8-439e-95a1-de2775a61e01 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:07.263987 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a619708c-c105-4ee0-abcb-4852e9cde729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:07.264593 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a619708c-c105-4ee0-abcb-4852e9cde729 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:07.264870 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a619708c-c105-4ee0-abcb-4852e9cde729 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:07.265141 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a619708c-c105-4ee0-abcb-4852e9cde729 admin admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:07.265782 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a619708c-c105-4ee0-abcb-4852e9cde729 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:07.266080 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a619708c-c105-4ee0-abcb-4852e9cde729 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:07.267958 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a619708c-c105-4ee0-abcb-4852e9cde729 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:07.279980 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1010/2047] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 24 17:28:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:07.284602 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d534af-5da9-4f29-a7ce-a7e86d8f39b1 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:07.295492 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d534af-5da9-4f29-a7ce-a7e86d8f39b1 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:07.301289 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6e6e8b-cda2-471b-a932-bdca191640b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:07.301892 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6e6e8b-cda2-471b-a932-bdca191640b6 admin admin] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:07.302182 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6e6e8b-cda2-471b-a932-bdca191640b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:07.302454 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6e6e8b-cda2-471b-a932-bdca191640b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:07.303142 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6e6e8b-cda2-471b-a932-bdca191640b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:07.305030 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6e6e8b-cda2-471b-a932-bdca191640b6 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:07.444647 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1038/2048] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 24 17:28:07 2022] POST /identity/v3/projects => generated 375 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:28:07.449572 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a619708c-c105-4ee0-abcb-4852e9cde729 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:07.459795 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a619708c-c105-4ee0-abcb-4852e9cde729 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:07.521637 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99a5bc8-f24b-45b8-8d01-1fba42ea9252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:07.522166 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99a5bc8-f24b-45b8-8d01-1fba42ea9252 admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:07.522443 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99a5bc8-f24b-45b8-8d01-1fba42ea9252 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:07.522712 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99a5bc8-f24b-45b8-8d01-1fba42ea9252 admin admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:07.523371 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99a5bc8-f24b-45b8-8d01-1fba42ea9252 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:07.525877 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99a5bc8-f24b-45b8-8d01-1fba42ea9252 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:07.644586 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:07.650235 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1011/2049] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 24 17:28:07 2022] POST /identity/v3/users => generated 288 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:28:07.654472 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6e6e8b-cda2-471b-a932-bdca191640b6 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:07.665536 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6e6e8b-cda2-471b-a932-bdca191640b6 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:07.670406 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4468e0-d4ef-43b6-8acc-c48f45c3788b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:07.671038 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4468e0-d4ef-43b6-8acc-c48f45c3788b admin admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:07.671320 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4468e0-d4ef-43b6-8acc-c48f45c3788b admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:07.671592 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4468e0-d4ef-43b6-8acc-c48f45c3788b admin admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:07.672239 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4468e0-d4ef-43b6-8acc-c48f45c3788b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:07.673817 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4468e0-d4ef-43b6-8acc-c48f45c3788b admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:07.683896 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1039/2050] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:28:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:28:07.688133 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99a5bc8-f24b-45b8-8d01-1fba42ea9252 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:07.698807 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99a5bc8-f24b-45b8-8d01-1fba42ea9252 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:07.703596 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d9d9b7-e773-444a-964e-cc8d18fed95b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:07.704201 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d9d9b7-e773-444a-964e-cc8d18fed95b admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:07.704482 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d9d9b7-e773-444a-964e-cc8d18fed95b admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:07.704753 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d9d9b7-e773-444a-964e-cc8d18fed95b admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:07.705388 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d9d9b7-e773-444a-964e-cc8d18fed95b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:07.707016 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d9d9b7-e773-444a-964e-cc8d18fed95b admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:07.718116 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1012/2051] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:28:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:28:07.723066 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4468e0-d4ef-43b6-8acc-c48f45c3788b admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:07.735744 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4468e0-d4ef-43b6-8acc-c48f45c3788b admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:07.741052 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b430c151-556e-496f-babb-4ab2808bfddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:07.741630 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b430c151-556e-496f-babb-4ab2808bfddd admin admin] REQUEST_METHOD: `PUT` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:07.741988 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b430c151-556e-496f-babb-4ab2808bfddd admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:07.742181 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b430c151-556e-496f-babb-4ab2808bfddd admin admin] PATH_INFO: `/v3/projects/de1ad2d3babc4f32af3f30a3dda9dd48/users/e40c080ddcf34210bcf1a75215ee1e43/roles/60dc7e16aee746d5a3e8aa27d7d8525c` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:07.758249 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b430c151-556e-496f-babb-4ab2808bfddd admin admin] RBAC: Authorizing `identity:create_grant(project_id=de1ad2d3babc4f32af3f30a3dda9dd48, user_id=e40c080ddcf34210bcf1a75215ee1e43, role_id=60dc7e16aee746d5a3e8aa27d7d8525c)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:07.760917 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b430c151-556e-496f-babb-4ab2808bfddd admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:07.879060 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1040/2052] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 24 17:28:07 2022] PUT /identity/v3/projects/de1ad2d3babc4f32af3f30a3dda9dd48/users/e40c080ddcf34210bcf1a75215ee1e43/roles/60dc7e16aee746d5a3e8aa27d7d8525c => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:28:07.890136 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1d2158-a9b0-4503-88d1-98a7e27b5435 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:07.890468 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1d2158-a9b0-4503-88d1-98a7e27b5435 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:07.890746 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1d2158-a9b0-4503-88d1-98a7e27b5435 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:07.925250 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-ed1d2158-a9b0-4503-88d1-98a7e27b5435 None None] MFA Rules not processed for user `e40c080ddcf34210bcf1a75215ee1e43`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:28:07.926496 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-ed1d2158-a9b0-4503-88d1-98a7e27b5435 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:28:07.997599 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1013/2053] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:28:07 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:28:08.015080 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b430c151-556e-496f-babb-4ab2808bfddd admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:08.017282 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-b430c151-556e-496f-babb-4ab2808bfddd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:28:08.079789 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b430c151-556e-496f-babb-4ab2808bfddd admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:08.085787 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b777e71-9b78-41e8-88b3-db71a50fe46c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be06246d47584b529150ceb98a5cfd95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:08.087163 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b777e71-9b78-41e8-88b3-db71a50fe46c service cinder] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:08.087448 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b777e71-9b78-41e8-88b3-db71a50fe46c service cinder] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:08.087725 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b777e71-9b78-41e8-88b3-db71a50fe46c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:08.096571 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b777e71-9b78-41e8-88b3-db71a50fe46c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:08.098023 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b777e71-9b78-41e8-88b3-db71a50fe46c service cinder] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:08.107799 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1041/2054] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:28:08 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:28:13.184022 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f791beb7-3fd9-40fa-9384-5aca18ea0abb None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:13.184022 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f791beb7-3fd9-40fa-9384-5aca18ea0abb None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:13.184022 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f791beb7-3fd9-40fa-9384-5aca18ea0abb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:13.221664 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-f791beb7-3fd9-40fa-9384-5aca18ea0abb None None] MFA Rules not processed for user `e40c080ddcf34210bcf1a75215ee1e43`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:28:13.224065 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-f791beb7-3fd9-40fa-9384-5aca18ea0abb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:28:13.233914 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1014/2055] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:28:13 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:28:17.661757 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eab2b6-ada8-44ec-8f1f-2b2482cfc796 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:17.662351 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eab2b6-ada8-44ec-8f1f-2b2482cfc796 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:17.662756 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eab2b6-ada8-44ec-8f1f-2b2482cfc796 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:17.693885 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-c2eab2b6-ada8-44ec-8f1f-2b2482cfc796 None None] MFA Rules not processed for user `64eaec18a9d94723b541497bb5e1c105`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:28:17.695253 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-c2eab2b6-ada8-44ec-8f1f-2b2482cfc796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:28:17.776079 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:17.788774 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1042/2056] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:28:17 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:28:18.651656 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f791beb7-3fd9-40fa-9384-5aca18ea0abb None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:18.653715 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-f791beb7-3fd9-40fa-9384-5aca18ea0abb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:28:18.745001 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f791beb7-3fd9-40fa-9384-5aca18ea0abb None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:18.750410 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b18453-7fda-46c9-8207-314acf8d50e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:18.751134 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b18453-7fda-46c9-8207-314acf8d50e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:18.751535 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b18453-7fda-46c9-8207-314acf8d50e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:18.751643 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b18453-7fda-46c9-8207-314acf8d50e2 admin admin] PATH_INFO: `/v3/users/f5ec7641876b4cf5b5d3585758502746` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:18.752768 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b18453-7fda-46c9-8207-314acf8d50e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f5ec7641876b4cf5b5d3585758502746)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:18.754758 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b18453-7fda-46c9-8207-314acf8d50e2 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:19.487519 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-58b18453-7fda-46c9-8207-314acf8d50e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f5ec7641876b4cf5b5d3585758502746'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:28:19.660268 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-58b18453-7fda-46c9-8207-314acf8d50e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f5ec7641876b4cf5b5d3585758502746'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:28:19.662597 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-58b18453-7fda-46c9-8207-314acf8d50e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f5ec7641876b4cf5b5d3585758502746'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:28:19.679255 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-58b18453-7fda-46c9-8207-314acf8d50e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f5ec7641876b4cf5b5d3585758502746'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:28:19.681537 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-58b18453-7fda-46c9-8207-314acf8d50e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f5ec7641876b4cf5b5d3585758502746'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:28:19.697983 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:19.716963 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1015/2057] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 24 17:28:18 2022] DELETE /identity/v3/users/f5ec7641876b4cf5b5d3585758502746 => generated 0 bytes in 1063 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:28:19.721283 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600e9706-285e-4b6f-958c-ff259debe647 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:19.721796 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600e9706-285e-4b6f-958c-ff259debe647 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:19.721877 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600e9706-285e-4b6f-958c-ff259debe647 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:19.755481 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-600e9706-285e-4b6f-958c-ff259debe647 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:28:19.756789 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-600e9706-285e-4b6f-958c-ff259debe647 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:28:19.840861 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1043/2058] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:28:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:28:19.850798 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b18453-7fda-46c9-8207-314acf8d50e2 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:19.852405 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-58b18453-7fda-46c9-8207-314acf8d50e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:28:19.935527 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b18453-7fda-46c9-8207-314acf8d50e2 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:19.940831 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5ce5bf-cb36-4edc-97f3-307e9a30e55e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e9ed4d47aef4140bfb2d3bb798cc559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:19.941986 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5ce5bf-cb36-4edc-97f3-307e9a30e55e service neutron] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:19.942291 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5ce5bf-cb36-4edc-97f3-307e9a30e55e service neutron] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:19.942862 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5ce5bf-cb36-4edc-97f3-307e9a30e55e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:19.957201 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5ce5bf-cb36-4edc-97f3-307e9a30e55e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:19.958218 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5ce5bf-cb36-4edc-97f3-307e9a30e55e service neutron] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:19.969993 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1016/2059] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:28:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:28:20.953520 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600e9706-285e-4b6f-958c-ff259debe647 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:20.955028 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-600e9706-285e-4b6f-958c-ff259debe647 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:28:20.968812 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600e9706-285e-4b6f-958c-ff259debe647 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:20.975217 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a2ddcc-14ba-4681-a07c-33410efb3252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:20.976103 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a2ddcc-14ba-4681-a07c-33410efb3252 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:20.976633 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a2ddcc-14ba-4681-a07c-33410efb3252 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:20.977635 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a2ddcc-14ba-4681-a07c-33410efb3252 admin admin] PATH_INFO: `/v3/projects/987b521e613c42f38771a63cc2345499` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:20.979166 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a2ddcc-14ba-4681-a07c-33410efb3252 admin admin] RBAC: Authorizing `identity:delete_project(project_id=987b521e613c42f38771a63cc2345499)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:20.981141 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a2ddcc-14ba-4681-a07c-33410efb3252 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:21.278767 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-b7a2ddcc-14ba-4681-a07c-33410efb3252 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '987b521e613c42f38771a63cc2345499', 'request_id': 'req-b7a2ddcc-14ba-4681-a07c-33410efb3252'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:28:21.422303 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-b7a2ddcc-14ba-4681-a07c-33410efb3252 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '987b521e613c42f38771a63cc2345499', 'request_id': 'req-b7a2ddcc-14ba-4681-a07c-33410efb3252'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:28:21.429940 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-b7a2ddcc-14ba-4681-a07c-33410efb3252 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '987b521e613c42f38771a63cc2345499', 'request_id': 'req-b7a2ddcc-14ba-4681-a07c-33410efb3252'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:28:21.441234 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-b7a2ddcc-14ba-4681-a07c-33410efb3252 admin admin] The token cache is being invalidate because project 987b521e613c42f38771a63cc2345499 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81731) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:28:21.441531 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-b7a2ddcc-14ba-4681-a07c-33410efb3252 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:28:21.442328 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-b7a2ddcc-14ba-4681-a07c-33410efb3252 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:28:21.443871 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1044/2060] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 24 17:28:20 2022] DELETE /identity/v3/projects/987b521e613c42f38771a63cc2345499 => generated 0 bytes in 491 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:28:21.448541 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5ce5bf-cb36-4edc-97f3-307e9a30e55e service neutron] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:21.458666 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5ce5bf-cb36-4edc-97f3-307e9a30e55e service neutron] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:21.529173 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12fd912-fbcd-4697-b860-3a61e7479168 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:21.529848 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12fd912-fbcd-4697-b860-3a61e7479168 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:21.530134 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12fd912-fbcd-4697-b860-3a61e7479168 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:21.530413 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12fd912-fbcd-4697-b860-3a61e7479168 admin admin] PATH_INFO: `/v3/users/809966567b8c464dbcf694da271b65da` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:21.531566 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12fd912-fbcd-4697-b860-3a61e7479168 admin admin] RBAC: Authorizing `identity:delete_user(user_id=809966567b8c464dbcf694da271b65da)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:21.533583 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12fd912-fbcd-4697-b860-3a61e7479168 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:21.886433 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-e12fd912-fbcd-4697-b860-3a61e7479168 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '809966567b8c464dbcf694da271b65da'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:28:22.042833 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-e12fd912-fbcd-4697-b860-3a61e7479168 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '809966567b8c464dbcf694da271b65da'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:28:22.043864 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-e12fd912-fbcd-4697-b860-3a61e7479168 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '809966567b8c464dbcf694da271b65da'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:28:22.052587 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-e12fd912-fbcd-4697-b860-3a61e7479168 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '809966567b8c464dbcf694da271b65da'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:28:22.053982 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-e12fd912-fbcd-4697-b860-3a61e7479168 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '809966567b8c464dbcf694da271b65da'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:28:22.079633 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1017/2061] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 24 17:28:21 2022] DELETE /identity/v3/users/809966567b8c464dbcf694da271b65da => generated 0 bytes in 632 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:28:22.731275 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a2ddcc-14ba-4681-a07c-33410efb3252 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:22.733559 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-b7a2ddcc-14ba-4681-a07c-33410efb3252 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:28:22.748595 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a2ddcc-14ba-4681-a07c-33410efb3252 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:22.840064 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dcc0cb-94f4-42a0-98d8-5aaaf7ea09b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:22.842020 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dcc0cb-94f4-42a0-98d8-5aaaf7ea09b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:22.842583 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dcc0cb-94f4-42a0-98d8-5aaaf7ea09b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:22.843070 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dcc0cb-94f4-42a0-98d8-5aaaf7ea09b5 admin admin] PATH_INFO: `/v3/projects/95d64b5d18d04a88a0723d2e3bf9a81d` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:22.850184 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dcc0cb-94f4-42a0-98d8-5aaaf7ea09b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=95d64b5d18d04a88a0723d2e3bf9a81d)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:22.850184 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dcc0cb-94f4-42a0-98d8-5aaaf7ea09b5 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:23.147957 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:23.166471 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-85dcc0cb-94f4-42a0-98d8-5aaaf7ea09b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95d64b5d18d04a88a0723d2e3bf9a81d', 'request_id': 'req-85dcc0cb-94f4-42a0-98d8-5aaaf7ea09b5'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:28:23.358121 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-85dcc0cb-94f4-42a0-98d8-5aaaf7ea09b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95d64b5d18d04a88a0723d2e3bf9a81d', 'request_id': 'req-85dcc0cb-94f4-42a0-98d8-5aaaf7ea09b5'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:28:23.368384 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-85dcc0cb-94f4-42a0-98d8-5aaaf7ea09b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95d64b5d18d04a88a0723d2e3bf9a81d', 'request_id': 'req-85dcc0cb-94f4-42a0-98d8-5aaaf7ea09b5'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:28:23.383782 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-85dcc0cb-94f4-42a0-98d8-5aaaf7ea09b5 admin admin] The token cache is being invalidate because project 95d64b5d18d04a88a0723d2e3bf9a81d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81731) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:28:23.384068 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-85dcc0cb-94f4-42a0-98d8-5aaaf7ea09b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:28:23.392776 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-85dcc0cb-94f4-42a0-98d8-5aaaf7ea09b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:28:23.392776 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1045/2062] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 24 17:28:22 2022] DELETE /identity/v3/projects/95d64b5d18d04a88a0723d2e3bf9a81d => generated 0 bytes in 657 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:28:23.394412 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04d1019-dd6a-41b5-a99c-33f46d264f88 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:23.394867 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04d1019-dd6a-41b5-a99c-33f46d264f88 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:23.395005 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04d1019-dd6a-41b5-a99c-33f46d264f88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:23.417031 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-b04d1019-dd6a-41b5-a99c-33f46d264f88 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:28:23.418375 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-b04d1019-dd6a-41b5-a99c-33f46d264f88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:28:23.428460 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:23.498179 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1018/2063] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:28:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:28:23.511752 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15f3038-4a50-4c3c-9591-eed6ef4d8167 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:23.512019 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15f3038-4a50-4c3c-9591-eed6ef4d8167 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:23.512243 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15f3038-4a50-4c3c-9591-eed6ef4d8167 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:23.536481 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-d15f3038-4a50-4c3c-9591-eed6ef4d8167 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:28:23.537756 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-d15f3038-4a50-4c3c-9591-eed6ef4d8167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:28:23.546980 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1046/2064] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:28:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:28:23.550948 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04d1019-dd6a-41b5-a99c-33f46d264f88 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:23.561044 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04d1019-dd6a-41b5-a99c-33f46d264f88 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:23.565957 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49db2a0-1e86-4a75-9496-fabbf4d3dd9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:23.566600 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49db2a0-1e86-4a75-9496-fabbf4d3dd9e admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:23.566879 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49db2a0-1e86-4a75-9496-fabbf4d3dd9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:23.567185 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49db2a0-1e86-4a75-9496-fabbf4d3dd9e admin admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:23.567874 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49db2a0-1e86-4a75-9496-fabbf4d3dd9e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:23.568207 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49db2a0-1e86-4a75-9496-fabbf4d3dd9e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:23.569827 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49db2a0-1e86-4a75-9496-fabbf4d3dd9e admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:23.589281 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1019/2065] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 24 17:28:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:28:23.596330 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15f3038-4a50-4c3c-9591-eed6ef4d8167 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:23.605787 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15f3038-4a50-4c3c-9591-eed6ef4d8167 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:23.610478 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b2238e-6a9b-499b-9018-bab231ba2f1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:23.611128 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b2238e-6a9b-499b-9018-bab231ba2f1a admin admin] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:23.611402 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b2238e-6a9b-499b-9018-bab231ba2f1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:23.611675 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b2238e-6a9b-499b-9018-bab231ba2f1a admin admin] PATH_INFO: `/v3/projects` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:23.612361 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b2238e-6a9b-499b-9018-bab231ba2f1a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:23.614263 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b2238e-6a9b-499b-9018-bab231ba2f1a admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:23.841351 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1047/2066] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 24 17:28:23 2022] POST /identity/v3/projects => generated 369 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:28:23.842607 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49db2a0-1e86-4a75-9496-fabbf4d3dd9e admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:23.858560 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49db2a0-1e86-4a75-9496-fabbf4d3dd9e admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:23.925411 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a83d982-33fd-4eba-a99e-125ac426a947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:23.926168 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a83d982-33fd-4eba-a99e-125ac426a947 admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:23.926451 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a83d982-33fd-4eba-a99e-125ac426a947 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:23.926726 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a83d982-33fd-4eba-a99e-125ac426a947 admin admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:23.928486 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a83d982-33fd-4eba-a99e-125ac426a947 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:23.933907 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a83d982-33fd-4eba-a99e-125ac426a947 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:24.169102 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1020/2067] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 24 17:28:23 2022] POST /identity/v3/users => generated 285 bytes in 327 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:28:24.173741 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b2238e-6a9b-499b-9018-bab231ba2f1a admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:24.186169 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b2238e-6a9b-499b-9018-bab231ba2f1a admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:24.191297 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b0f905-93a7-4c79-9517-07adcf75825e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:24.191922 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b0f905-93a7-4c79-9517-07adcf75825e admin admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:24.192202 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b0f905-93a7-4c79-9517-07adcf75825e admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:24.192484 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b0f905-93a7-4c79-9517-07adcf75825e admin admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:24.193130 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b0f905-93a7-4c79-9517-07adcf75825e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:24.195291 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b0f905-93a7-4c79-9517-07adcf75825e admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:24.208760 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1048/2068] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:28:24 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:28:24.213046 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a83d982-33fd-4eba-a99e-125ac426a947 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:24.224981 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a83d982-33fd-4eba-a99e-125ac426a947 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:24.229668 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464affd8-efdb-438a-8706-fcc508b1c053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:24.230276 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464affd8-efdb-438a-8706-fcc508b1c053 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:24.230550 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464affd8-efdb-438a-8706-fcc508b1c053 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:24.230816 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464affd8-efdb-438a-8706-fcc508b1c053 admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:24.231482 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464affd8-efdb-438a-8706-fcc508b1c053 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:24.233103 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464affd8-efdb-438a-8706-fcc508b1c053 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:24.244874 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1021/2069] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:28:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:28:24.249504 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b0f905-93a7-4c79-9517-07adcf75825e admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:24.261598 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b0f905-93a7-4c79-9517-07adcf75825e admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:24.269339 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73df33e-7de0-43ba-a52e-7570beea1c4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:24.275644 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73df33e-7de0-43ba-a52e-7570beea1c4e admin admin] REQUEST_METHOD: `PUT` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:24.275644 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73df33e-7de0-43ba-a52e-7570beea1c4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:24.275644 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73df33e-7de0-43ba-a52e-7570beea1c4e admin admin] PATH_INFO: `/v3/projects/15b6ff2ca77649dda0b59108c5c70745/users/64bd893a8ab741c086e223c7ff68a13b/roles/60dc7e16aee746d5a3e8aa27d7d8525c` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:24.287764 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73df33e-7de0-43ba-a52e-7570beea1c4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=15b6ff2ca77649dda0b59108c5c70745, user_id=64bd893a8ab741c086e223c7ff68a13b, role_id=60dc7e16aee746d5a3e8aa27d7d8525c)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:24.290974 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73df33e-7de0-43ba-a52e-7570beea1c4e admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:24.511134 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1049/2070] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 24 17:28:24 2022] PUT /identity/v3/projects/15b6ff2ca77649dda0b59108c5c70745/users/64bd893a8ab741c086e223c7ff68a13b/roles/60dc7e16aee746d5a3e8aa27d7d8525c => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:28:24.524796 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64adef4f-585c-426e-93f6-7e78afded7c6 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:24.525690 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64adef4f-585c-426e-93f6-7e78afded7c6 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:24.525965 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64adef4f-585c-426e-93f6-7e78afded7c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:24.568095 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-64adef4f-585c-426e-93f6-7e78afded7c6 None None] MFA Rules not processed for user `64bd893a8ab741c086e223c7ff68a13b`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:28:24.569868 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-64adef4f-585c-426e-93f6-7e78afded7c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:28:24.655671 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1022/2071] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:28:24 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:28:24.660110 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73df33e-7de0-43ba-a52e-7570beea1c4e admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:24.727858 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73df33e-7de0-43ba-a52e-7570beea1c4e admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:24.732546 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4543858a-8daa-4ab3-aadf-de221b720369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:24.733166 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4543858a-8daa-4ab3-aadf-de221b720369 admin admin] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:24.733445 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4543858a-8daa-4ab3-aadf-de221b720369 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:24.733717 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4543858a-8daa-4ab3-aadf-de221b720369 admin admin] PATH_INFO: `/v3/projects` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:24.734400 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4543858a-8daa-4ab3-aadf-de221b720369 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:24.736388 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4543858a-8daa-4ab3-aadf-de221b720369 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:24.964655 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1050/2072] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 24 17:28:24 2022] POST /identity/v3/projects => generated 367 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:28:24.969290 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64adef4f-585c-426e-93f6-7e78afded7c6 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:24.980162 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64adef4f-585c-426e-93f6-7e78afded7c6 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:25.065297 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf75da94-a463-4248-b897-a2870e6a5ce1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:25.066212 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf75da94-a463-4248-b897-a2870e6a5ce1 admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:25.066446 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf75da94-a463-4248-b897-a2870e6a5ce1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:25.066723 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf75da94-a463-4248-b897-a2870e6a5ce1 admin admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:25.067412 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf75da94-a463-4248-b897-a2870e6a5ce1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:25.069341 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf75da94-a463-4248-b897-a2870e6a5ce1 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:25.277449 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1023/2073] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 24 17:28:24 2022] POST /identity/v3/users => generated 290 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:28:25.281675 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4543858a-8daa-4ab3-aadf-de221b720369 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:25.292652 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4543858a-8daa-4ab3-aadf-de221b720369 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:25.297224 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c5c9b2-e46f-459b-9fb8-56b0bac96867 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:25.297822 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c5c9b2-e46f-459b-9fb8-56b0bac96867 admin admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:25.298114 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c5c9b2-e46f-459b-9fb8-56b0bac96867 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:25.298440 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c5c9b2-e46f-459b-9fb8-56b0bac96867 admin admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:25.299116 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c5c9b2-e46f-459b-9fb8-56b0bac96867 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:25.300719 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c5c9b2-e46f-459b-9fb8-56b0bac96867 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:25.313342 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1051/2074] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:28:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:28:25.317759 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf75da94-a463-4248-b897-a2870e6a5ce1 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:25.328092 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf75da94-a463-4248-b897-a2870e6a5ce1 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:25.332886 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da0093b-9339-4683-8d4e-d9efa809eb63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:25.333550 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da0093b-9339-4683-8d4e-d9efa809eb63 admin admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:25.333837 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da0093b-9339-4683-8d4e-d9efa809eb63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:25.334112 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da0093b-9339-4683-8d4e-d9efa809eb63 admin admin] PATH_INFO: `/v3/projects/37ab072bdcc44689bf8fa86a27e25396/users/80932420e1fd4df6bc461fdde977e98e/roles/c5346bc54abf445f9900edf94264c850` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:25.349108 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da0093b-9339-4683-8d4e-d9efa809eb63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=37ab072bdcc44689bf8fa86a27e25396, user_id=80932420e1fd4df6bc461fdde977e98e, role_id=c5346bc54abf445f9900edf94264c850)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:25.352396 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da0093b-9339-4683-8d4e-d9efa809eb63 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:25.403366 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:25.530128 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1024/2075] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 24 17:28:25 2022] PUT /identity/v3/projects/37ab072bdcc44689bf8fa86a27e25396/users/80932420e1fd4df6bc461fdde977e98e/roles/c5346bc54abf445f9900edf94264c850 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:28:25.534881 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c5c9b2-e46f-459b-9fb8-56b0bac96867 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:25.545739 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c5c9b2-e46f-459b-9fb8-56b0bac96867 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:25.575936 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:25.642593 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb0d103-09b3-4b9e-a947-7ed9efd809f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:25.642593 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb0d103-09b3-4b9e-a947-7ed9efd809f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:25.642593 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb0d103-09b3-4b9e-a947-7ed9efd809f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:25.642593 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb0d103-09b3-4b9e-a947-7ed9efd809f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:25.643467 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb0d103-09b3-4b9e-a947-7ed9efd809f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:25.643467 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb0d103-09b3-4b9e-a947-7ed9efd809f4 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:25.643984 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1052/2076] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:28:25 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:28:25.649021 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da0093b-9339-4683-8d4e-d9efa809eb63 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:25.661066 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da0093b-9339-4683-8d4e-d9efa809eb63 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:25.667394 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8b5142-238a-40fd-9f69-eecc9b3f3dd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:25.668564 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8b5142-238a-40fd-9f69-eecc9b3f3dd2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:25.669351 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8b5142-238a-40fd-9f69-eecc9b3f3dd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:25.669514 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8b5142-238a-40fd-9f69-eecc9b3f3dd2 admin admin] PATH_INFO: `/v3/projects/37ab072bdcc44689bf8fa86a27e25396/users/80932420e1fd4df6bc461fdde977e98e/roles/c5346bc54abf445f9900edf94264c850` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:25.671787 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8b5142-238a-40fd-9f69-eecc9b3f3dd2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=37ab072bdcc44689bf8fa86a27e25396, user_id=80932420e1fd4df6bc461fdde977e98e, role_id=c5346bc54abf445f9900edf94264c850)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:25.674747 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8b5142-238a-40fd-9f69-eecc9b3f3dd2 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:25.703378 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1025/2077] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 24 17:28:25 2022] PUT /identity/v3/projects/37ab072bdcc44689bf8fa86a27e25396/users/80932420e1fd4df6bc461fdde977e98e/roles/c5346bc54abf445f9900edf94264c850 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:28:25.707859 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192869b8-ae82-4734-8e90-20562e7f0141 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:25.708217 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192869b8-ae82-4734-8e90-20562e7f0141 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:25.708535 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192869b8-ae82-4734-8e90-20562e7f0141 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:25.750513 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-192869b8-ae82-4734-8e90-20562e7f0141 None None] MFA Rules not processed for user `80932420e1fd4df6bc461fdde977e98e`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:28:25.751814 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-192869b8-ae82-4734-8e90-20562e7f0141 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:28:25.843965 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1053/2078] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:28:25 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:28:25.876147 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8b5142-238a-40fd-9f69-eecc9b3f3dd2 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:25.878428 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-bd8b5142-238a-40fd-9f69-eecc9b3f3dd2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:28:25.959644 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8b5142-238a-40fd-9f69-eecc9b3f3dd2 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:25.964838 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44df5e89-819e-47a0-b993-0443c043aeb6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be06246d47584b529150ceb98a5cfd95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:25.968187 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44df5e89-819e-47a0-b993-0443c043aeb6 service cinder] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:25.968187 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44df5e89-819e-47a0-b993-0443c043aeb6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:25.968187 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44df5e89-819e-47a0-b993-0443c043aeb6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:25.982421 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44df5e89-819e-47a0-b993-0443c043aeb6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:25.983913 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44df5e89-819e-47a0-b993-0443c043aeb6 service cinder] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:25.994126 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1026/2079] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:28:25 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:28:29.417284 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853c28dc-0d4d-4a32-a82f-5a19ab2d8c8e None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:29.417284 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853c28dc-0d4d-4a32-a82f-5a19ab2d8c8e None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:29.417284 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853c28dc-0d4d-4a32-a82f-5a19ab2d8c8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:29.434291 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-853c28dc-0d4d-4a32-a82f-5a19ab2d8c8e None None] MFA Rules not processed for user `47da700d7d0f4d5cb1ea984a82d643bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:28:29.435743 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-853c28dc-0d4d-4a32-a82f-5a19ab2d8c8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:28:29.521332 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1054/2080] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:28:29 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:28:30.203384 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44df5e89-819e-47a0-b993-0443c043aeb6 service cinder] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:30.285612 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44df5e89-819e-47a0-b993-0443c043aeb6 service cinder] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:30.293899 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f29fb53-45a1-4a6d-8bc3-bc2263a3a2ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:30.294555 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f29fb53-45a1-4a6d-8bc3-bc2263a3a2ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:30.294832 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f29fb53-45a1-4a6d-8bc3-bc2263a3a2ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:30.295332 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f29fb53-45a1-4a6d-8bc3-bc2263a3a2ee admin admin] PATH_INFO: `/v3/users/64bd893a8ab741c086e223c7ff68a13b` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:30.297848 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f29fb53-45a1-4a6d-8bc3-bc2263a3a2ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=64bd893a8ab741c086e223c7ff68a13b)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:30.304455 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f29fb53-45a1-4a6d-8bc3-bc2263a3a2ee admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:30.316134 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:30.585007 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-4f29fb53-45a1-4a6d-8bc3-bc2263a3a2ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64bd893a8ab741c086e223c7ff68a13b'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:28:30.726567 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-4f29fb53-45a1-4a6d-8bc3-bc2263a3a2ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64bd893a8ab741c086e223c7ff68a13b'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:28:30.727354 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-4f29fb53-45a1-4a6d-8bc3-bc2263a3a2ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64bd893a8ab741c086e223c7ff68a13b'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:28:30.735476 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-4f29fb53-45a1-4a6d-8bc3-bc2263a3a2ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64bd893a8ab741c086e223c7ff68a13b'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:28:30.736138 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-4f29fb53-45a1-4a6d-8bc3-bc2263a3a2ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64bd893a8ab741c086e223c7ff68a13b'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:28:30.761910 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1027/2081] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 24 17:28:30 2022] DELETE /identity/v3/users/64bd893a8ab741c086e223c7ff68a13b => generated 0 bytes in 559 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:28:30.766452 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108945bb-90cd-4d92-b82f-b7fa79d8c77c None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:30.767434 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108945bb-90cd-4d92-b82f-b7fa79d8c77c None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:30.767745 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108945bb-90cd-4d92-b82f-b7fa79d8c77c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:30.790472 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-108945bb-90cd-4d92-b82f-b7fa79d8c77c None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:28:30.791754 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-108945bb-90cd-4d92-b82f-b7fa79d8c77c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:28:30.853532 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:30.871600 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1055/2082] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:28:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:28:30.882795 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f29fb53-45a1-4a6d-8bc3-bc2263a3a2ee admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:30.885999 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-4f29fb53-45a1-4a6d-8bc3-bc2263a3a2ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:28:30.962938 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f29fb53-45a1-4a6d-8bc3-bc2263a3a2ee admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:30.966749 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b83d3b8-636e-4c23-9b50-fff7c192110c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e9ed4d47aef4140bfb2d3bb798cc559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:30.967417 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b83d3b8-636e-4c23-9b50-fff7c192110c service neutron] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:30.967697 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b83d3b8-636e-4c23-9b50-fff7c192110c service neutron] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:30.967973 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b83d3b8-636e-4c23-9b50-fff7c192110c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:30.978265 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b83d3b8-636e-4c23-9b50-fff7c192110c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:30.979754 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b83d3b8-636e-4c23-9b50-fff7c192110c service neutron] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:30.989847 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1028/2083] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:28:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:28:31.679739 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108945bb-90cd-4d92-b82f-b7fa79d8c77c None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:31.682075 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-108945bb-90cd-4d92-b82f-b7fa79d8c77c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:28:31.695536 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108945bb-90cd-4d92-b82f-b7fa79d8c77c None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:31.704640 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e10902f-615c-467b-b620-c45689adf26b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:31.705348 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e10902f-615c-467b-b620-c45689adf26b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:31.705815 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e10902f-615c-467b-b620-c45689adf26b admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:31.706255 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e10902f-615c-467b-b620-c45689adf26b admin admin] PATH_INFO: `/v3/projects/15b6ff2ca77649dda0b59108c5c70745` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:31.707651 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e10902f-615c-467b-b620-c45689adf26b admin admin] RBAC: Authorizing `identity:delete_project(project_id=15b6ff2ca77649dda0b59108c5c70745)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:31.710007 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e10902f-615c-467b-b620-c45689adf26b admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:31.851731 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-1e10902f-615c-467b-b620-c45689adf26b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '15b6ff2ca77649dda0b59108c5c70745', 'request_id': 'req-1e10902f-615c-467b-b620-c45689adf26b'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:28:31.946403 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-1e10902f-615c-467b-b620-c45689adf26b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '15b6ff2ca77649dda0b59108c5c70745', 'request_id': 'req-1e10902f-615c-467b-b620-c45689adf26b'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:28:31.955130 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-1e10902f-615c-467b-b620-c45689adf26b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '15b6ff2ca77649dda0b59108c5c70745', 'request_id': 'req-1e10902f-615c-467b-b620-c45689adf26b'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:28:31.973009 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-1e10902f-615c-467b-b620-c45689adf26b admin admin] The token cache is being invalidate because project 15b6ff2ca77649dda0b59108c5c70745 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81731) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:28:31.973520 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-1e10902f-615c-467b-b620-c45689adf26b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:28:31.974232 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-1e10902f-615c-467b-b620-c45689adf26b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:28:31.976516 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1056/2084] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 24 17:28:31 2022] DELETE /identity/v3/projects/15b6ff2ca77649dda0b59108c5c70745 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:28:31.981384 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b83d3b8-636e-4c23-9b50-fff7c192110c service neutron] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:31.991601 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b83d3b8-636e-4c23-9b50-fff7c192110c service neutron] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:32.052151 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d9664b-ae63-43f5-bcad-8108c04a3a03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:32.052806 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d9664b-ae63-43f5-bcad-8108c04a3a03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:32.053109 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d9664b-ae63-43f5-bcad-8108c04a3a03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:32.053348 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d9664b-ae63-43f5-bcad-8108c04a3a03 admin admin] PATH_INFO: `/v3/users/80932420e1fd4df6bc461fdde977e98e` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:32.054536 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d9664b-ae63-43f5-bcad-8108c04a3a03 admin admin] RBAC: Authorizing `identity:delete_user(user_id=80932420e1fd4df6bc461fdde977e98e)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:32.056564 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d9664b-ae63-43f5-bcad-8108c04a3a03 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:32.222799 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:32.377184 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-01d9664b-ae63-43f5-bcad-8108c04a3a03 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '80932420e1fd4df6bc461fdde977e98e'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:28:32.482740 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-01d9664b-ae63-43f5-bcad-8108c04a3a03 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '80932420e1fd4df6bc461fdde977e98e'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:28:32.483792 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-01d9664b-ae63-43f5-bcad-8108c04a3a03 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '80932420e1fd4df6bc461fdde977e98e'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:28:32.493947 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-01d9664b-ae63-43f5-bcad-8108c04a3a03 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '80932420e1fd4df6bc461fdde977e98e'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:28:32.497759 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-01d9664b-ae63-43f5-bcad-8108c04a3a03 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '80932420e1fd4df6bc461fdde977e98e'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:28:32.515277 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1029/2085] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 24 17:28:31 2022] DELETE /identity/v3/users/80932420e1fd4df6bc461fdde977e98e => generated 0 bytes in 534 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:28:33.179358 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e10902f-615c-467b-b620-c45689adf26b admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:33.184859 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-1e10902f-615c-467b-b620-c45689adf26b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:28:33.208647 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e10902f-615c-467b-b620-c45689adf26b admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:33.290330 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e7aba0-8c55-48f3-ad42-3432806b6edb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:33.291509 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e7aba0-8c55-48f3-ad42-3432806b6edb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:33.292473 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e7aba0-8c55-48f3-ad42-3432806b6edb admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:33.292995 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e7aba0-8c55-48f3-ad42-3432806b6edb admin admin] PATH_INFO: `/v3/projects/37ab072bdcc44689bf8fa86a27e25396` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:33.295701 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e7aba0-8c55-48f3-ad42-3432806b6edb admin admin] RBAC: Authorizing `identity:delete_project(project_id=37ab072bdcc44689bf8fa86a27e25396)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:33.298311 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e7aba0-8c55-48f3-ad42-3432806b6edb admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:33.487663 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-97e7aba0-8c55-48f3-ad42-3432806b6edb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37ab072bdcc44689bf8fa86a27e25396', 'request_id': 'req-97e7aba0-8c55-48f3-ad42-3432806b6edb'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:28:33.619386 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-97e7aba0-8c55-48f3-ad42-3432806b6edb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37ab072bdcc44689bf8fa86a27e25396', 'request_id': 'req-97e7aba0-8c55-48f3-ad42-3432806b6edb'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:28:33.627761 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-97e7aba0-8c55-48f3-ad42-3432806b6edb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37ab072bdcc44689bf8fa86a27e25396', 'request_id': 'req-97e7aba0-8c55-48f3-ad42-3432806b6edb'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:28:33.640178 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-97e7aba0-8c55-48f3-ad42-3432806b6edb admin admin] The token cache is being invalidate because project 37ab072bdcc44689bf8fa86a27e25396 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81731) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:28:33.640556 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-97e7aba0-8c55-48f3-ad42-3432806b6edb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:28:33.641256 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-97e7aba0-8c55-48f3-ad42-3432806b6edb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:28:33.643286 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1057/2086] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 24 17:28:33 2022] DELETE /identity/v3/projects/37ab072bdcc44689bf8fa86a27e25396 => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:28:33.648128 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefdb68b-1538-4d67-97f7-0603267da04c None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:33.648592 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefdb68b-1538-4d67-97f7-0603267da04c None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:33.649369 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefdb68b-1538-4d67-97f7-0603267da04c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:33.671259 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-eefdb68b-1538-4d67-97f7-0603267da04c None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:28:33.677502 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-eefdb68b-1538-4d67-97f7-0603267da04c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:28:33.743026 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1030/2087] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:28:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:28:33.750976 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891ced8b-06b3-4f70-a108-1f4dbec1717e None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:33.751269 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891ced8b-06b3-4f70-a108-1f4dbec1717e None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:33.751562 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891ced8b-06b3-4f70-a108-1f4dbec1717e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:33.760833 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:33.772252 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-891ced8b-06b3-4f70-a108-1f4dbec1717e None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:28:33.773464 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-891ced8b-06b3-4f70-a108-1f4dbec1717e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:28:33.782295 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1058/2088] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:28:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:28:33.787115 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefdb68b-1538-4d67-97f7-0603267da04c None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:33.796922 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefdb68b-1538-4d67-97f7-0603267da04c None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:33.801626 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6a92cb-95f6-4365-ab31-61f20ff1262f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:33.802231 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6a92cb-95f6-4365-ab31-61f20ff1262f admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:33.802507 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6a92cb-95f6-4365-ab31-61f20ff1262f admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:33.802781 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6a92cb-95f6-4365-ab31-61f20ff1262f admin admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:33.803465 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6a92cb-95f6-4365-ab31-61f20ff1262f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:28:33.803770 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6a92cb-95f6-4365-ab31-61f20ff1262f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:33.805901 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6a92cb-95f6-4365-ab31-61f20ff1262f admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:33.818167 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1031/2089] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 24 17:28:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:28:33.823357 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891ced8b-06b3-4f70-a108-1f4dbec1717e None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:33.832776 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891ced8b-06b3-4f70-a108-1f4dbec1717e None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:33.837643 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da553ad6-7cf8-4b23-aec6-300e757a4ab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:33.838238 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da553ad6-7cf8-4b23-aec6-300e757a4ab9 admin admin] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:33.838519 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da553ad6-7cf8-4b23-aec6-300e757a4ab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:33.838792 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da553ad6-7cf8-4b23-aec6-300e757a4ab9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:33.839458 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da553ad6-7cf8-4b23-aec6-300e757a4ab9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:33.841515 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da553ad6-7cf8-4b23-aec6-300e757a4ab9 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:33.964047 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1059/2090] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 24 17:28:33 2022] POST /identity/v3/projects => generated 375 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:28:33.968239 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6a92cb-95f6-4365-ab31-61f20ff1262f admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:33.978870 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6a92cb-95f6-4365-ab31-61f20ff1262f admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:34.042789 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fb291d-e75d-439b-8dd3-f497120ef1f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:34.043348 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fb291d-e75d-439b-8dd3-f497120ef1f3 admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:34.043629 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fb291d-e75d-439b-8dd3-f497120ef1f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:34.043900 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fb291d-e75d-439b-8dd3-f497120ef1f3 admin admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:34.044525 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fb291d-e75d-439b-8dd3-f497120ef1f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:34.046458 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fb291d-e75d-439b-8dd3-f497120ef1f3 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:34.214601 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1032/2091] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 24 17:28:33 2022] POST /identity/v3/users => generated 288 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:28:34.218710 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da553ad6-7cf8-4b23-aec6-300e757a4ab9 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:34.229933 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da553ad6-7cf8-4b23-aec6-300e757a4ab9 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:34.234531 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c488dc2-1f28-47d9-a381-342c368fe3c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:34.235176 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c488dc2-1f28-47d9-a381-342c368fe3c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:34.235450 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c488dc2-1f28-47d9-a381-342c368fe3c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:34.235721 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c488dc2-1f28-47d9-a381-342c368fe3c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:34.236363 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c488dc2-1f28-47d9-a381-342c368fe3c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:34.237947 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c488dc2-1f28-47d9-a381-342c368fe3c6 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:34.248705 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1060/2092] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:28:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:28:34.252823 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fb291d-e75d-439b-8dd3-f497120ef1f3 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:34.263056 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fb291d-e75d-439b-8dd3-f497120ef1f3 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:34.268031 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6244785c-423b-4a4a-9555-0af28615e5f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:34.268650 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6244785c-423b-4a4a-9555-0af28615e5f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:34.268908 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6244785c-423b-4a4a-9555-0af28615e5f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:34.269223 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6244785c-423b-4a4a-9555-0af28615e5f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:34.270191 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6244785c-423b-4a4a-9555-0af28615e5f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:34.271830 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6244785c-423b-4a4a-9555-0af28615e5f4 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:34.282641 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1033/2093] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:28:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:28:34.286915 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c488dc2-1f28-47d9-a381-342c368fe3c6 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:34.297538 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c488dc2-1f28-47d9-a381-342c368fe3c6 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:34.302672 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e6d5f2-cce4-4eae-b776-71ff9eb7a12a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:34.303362 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e6d5f2-cce4-4eae-b776-71ff9eb7a12a admin admin] REQUEST_METHOD: `PUT` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:34.303729 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e6d5f2-cce4-4eae-b776-71ff9eb7a12a admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:34.304006 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e6d5f2-cce4-4eae-b776-71ff9eb7a12a admin admin] PATH_INFO: `/v3/projects/e58e1abd4faf4854b65a7d284c588562/users/3fa20a7dc3a64475a7c55057a81fb371/roles/60dc7e16aee746d5a3e8aa27d7d8525c` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:34.319683 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e6d5f2-cce4-4eae-b776-71ff9eb7a12a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e58e1abd4faf4854b65a7d284c588562, user_id=3fa20a7dc3a64475a7c55057a81fb371, role_id=60dc7e16aee746d5a3e8aa27d7d8525c)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:34.322318 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e6d5f2-cce4-4eae-b776-71ff9eb7a12a admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:34.455143 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1061/2094] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 24 17:28:34 2022] PUT /identity/v3/projects/e58e1abd4faf4854b65a7d284c588562/users/3fa20a7dc3a64475a7c55057a81fb371/roles/60dc7e16aee746d5a3e8aa27d7d8525c => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:28:34.466941 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65683610-f450-4890-b59f-32f7beb0f64a None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:34.467234 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65683610-f450-4890-b59f-32f7beb0f64a None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:34.467509 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65683610-f450-4890-b59f-32f7beb0f64a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:34.500017 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-65683610-f450-4890-b59f-32f7beb0f64a None None] MFA Rules not processed for user `3fa20a7dc3a64475a7c55057a81fb371`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:28:34.501303 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-65683610-f450-4890-b59f-32f7beb0f64a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:28:34.522972 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:34.593925 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1034/2095] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:28:34 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:28:34.602533 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e6d5f2-cce4-4eae-b776-71ff9eb7a12a admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:34.715465 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e6d5f2-cce4-4eae-b776-71ff9eb7a12a admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:34.723561 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eff8fa-0429-49c3-a1e2-3cb2e254e2be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:34.724632 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eff8fa-0429-49c3-a1e2-3cb2e254e2be admin admin] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:34.725140 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eff8fa-0429-49c3-a1e2-3cb2e254e2be admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:34.725657 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eff8fa-0429-49c3-a1e2-3cb2e254e2be admin admin] PATH_INFO: `/v3/projects` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:34.726823 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83eff8fa-0429-49c3-a1e2-3cb2e254e2be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:34.730378 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83eff8fa-0429-49c3-a1e2-3cb2e254e2be admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:34.867090 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1062/2096] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 24 17:28:34 2022] POST /identity/v3/projects => generated 371 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:28:34.871554 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65683610-f450-4890-b59f-32f7beb0f64a None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:34.882484 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65683610-f450-4890-b59f-32f7beb0f64a None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:34.951370 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737d53f4-087d-44d1-b669-a0b3f3481d53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:34.951991 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737d53f4-087d-44d1-b669-a0b3f3481d53 admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:34.952541 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737d53f4-087d-44d1-b669-a0b3f3481d53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:34.952831 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737d53f4-087d-44d1-b669-a0b3f3481d53 admin admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:34.953470 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737d53f4-087d-44d1-b669-a0b3f3481d53 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:34.955410 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737d53f4-087d-44d1-b669-a0b3f3481d53 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:35.103018 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1035/2097] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 24 17:28:34 2022] POST /identity/v3/users => generated 292 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:28:35.104351 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eff8fa-0429-49c3-a1e2-3cb2e254e2be admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:35.114828 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eff8fa-0429-49c3-a1e2-3cb2e254e2be admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:35.119655 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0fc6af-f65d-44e2-bdb1-60afd3af4b53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:35.120277 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0fc6af-f65d-44e2-bdb1-60afd3af4b53 admin admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:35.120671 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0fc6af-f65d-44e2-bdb1-60afd3af4b53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:35.121035 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0fc6af-f65d-44e2-bdb1-60afd3af4b53 admin admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:35.121689 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0fc6af-f65d-44e2-bdb1-60afd3af4b53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:35.123343 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0fc6af-f65d-44e2-bdb1-60afd3af4b53 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:35.136496 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1063/2098] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:28:35 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:28:35.144759 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737d53f4-087d-44d1-b669-a0b3f3481d53 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:35.156602 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737d53f4-087d-44d1-b669-a0b3f3481d53 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:35.161330 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5bae25-c2ff-475d-ab03-478408b11c16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:35.161982 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5bae25-c2ff-475d-ab03-478408b11c16 admin admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:35.162288 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5bae25-c2ff-475d-ab03-478408b11c16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:35.162560 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5bae25-c2ff-475d-ab03-478408b11c16 admin admin] PATH_INFO: `/v3/projects/5b07a5c1f36f43658781f8c330dbb9c9/users/629e894226864c40adc39103b81ec802/roles/c5346bc54abf445f9900edf94264c850` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:35.177910 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5bae25-c2ff-475d-ab03-478408b11c16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b07a5c1f36f43658781f8c330dbb9c9, user_id=629e894226864c40adc39103b81ec802, role_id=c5346bc54abf445f9900edf94264c850)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:35.181136 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5bae25-c2ff-475d-ab03-478408b11c16 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:35.306074 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1036/2099] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 24 17:28:35 2022] PUT /identity/v3/projects/5b07a5c1f36f43658781f8c330dbb9c9/users/629e894226864c40adc39103b81ec802/roles/c5346bc54abf445f9900edf94264c850 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:28:35.310188 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0fc6af-f65d-44e2-bdb1-60afd3af4b53 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:35.321145 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0fc6af-f65d-44e2-bdb1-60afd3af4b53 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:35.379318 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5765ff-8228-4d25-b766-772b86bd2804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:35.380769 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5765ff-8228-4d25-b766-772b86bd2804 admin admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:35.381050 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5765ff-8228-4d25-b766-772b86bd2804 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:35.381362 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5765ff-8228-4d25-b766-772b86bd2804 admin admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:35.382104 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5765ff-8228-4d25-b766-772b86bd2804 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:35.384366 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5765ff-8228-4d25-b766-772b86bd2804 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:35.395098 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1064/2100] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:28:35 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:28:35.399799 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5bae25-c2ff-475d-ab03-478408b11c16 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:35.410145 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5bae25-c2ff-475d-ab03-478408b11c16 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:35.415026 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0455f488-6ecf-4d7e-a1f7-971833e52e1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:35.415683 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0455f488-6ecf-4d7e-a1f7-971833e52e1d admin admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:35.415960 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0455f488-6ecf-4d7e-a1f7-971833e52e1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:35.416237 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0455f488-6ecf-4d7e-a1f7-971833e52e1d admin admin] PATH_INFO: `/v3/projects/5b07a5c1f36f43658781f8c330dbb9c9/users/629e894226864c40adc39103b81ec802/roles/c5346bc54abf445f9900edf94264c850` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:35.418206 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0455f488-6ecf-4d7e-a1f7-971833e52e1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b07a5c1f36f43658781f8c330dbb9c9, user_id=629e894226864c40adc39103b81ec802, role_id=c5346bc54abf445f9900edf94264c850)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:35.420874 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0455f488-6ecf-4d7e-a1f7-971833e52e1d admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:35.432985 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1037/2101] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 24 17:28:35 2022] PUT /identity/v3/projects/5b07a5c1f36f43658781f8c330dbb9c9/users/629e894226864c40adc39103b81ec802/roles/c5346bc54abf445f9900edf94264c850 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:28:35.445162 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b2d9a9-e655-46f6-8307-7d776111fabd None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:35.445448 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b2d9a9-e655-46f6-8307-7d776111fabd None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:35.446267 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b2d9a9-e655-46f6-8307-7d776111fabd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:35.466299 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:35.473612 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-97b2d9a9-e655-46f6-8307-7d776111fabd None None] MFA Rules not processed for user `629e894226864c40adc39103b81ec802`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:28:35.474797 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-97b2d9a9-e655-46f6-8307-7d776111fabd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:28:35.548029 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1065/2102] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:28:35 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:28:35.562331 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0455f488-6ecf-4d7e-a1f7-971833e52e1d admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:35.562331 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-0455f488-6ecf-4d7e-a1f7-971833e52e1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:28:35.621014 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0455f488-6ecf-4d7e-a1f7-971833e52e1d admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:35.625199 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc046df-ea83-4daf-89eb-944d46024d68 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be06246d47584b529150ceb98a5cfd95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:35.625778 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc046df-ea83-4daf-89eb-944d46024d68 service cinder] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:35.626059 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc046df-ea83-4daf-89eb-944d46024d68 service cinder] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:35.626335 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc046df-ea83-4daf-89eb-944d46024d68 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:35.632434 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:35.704725 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc046df-ea83-4daf-89eb-944d46024d68 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:35.706527 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc046df-ea83-4daf-89eb-944d46024d68 service cinder] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:35.717521 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1038/2103] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:28:35 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:28:44.739048 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b2d9a9-e655-46f6-8307-7d776111fabd None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:28:44.749363 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b2d9a9-e655-46f6-8307-7d776111fabd None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:28:44.753223 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1eec9e1-1f49-4030-bfee-e515aea42ff2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be06246d47584b529150ceb98a5cfd95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:28:44.753766 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eec9e1-1f49-4030-bfee-e515aea42ff2 service cinder] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:28:44.754044 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eec9e1-1f49-4030-bfee-e515aea42ff2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:28:44.754315 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eec9e1-1f49-4030-bfee-e515aea42ff2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:28:44.764212 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1eec9e1-1f49-4030-bfee-e515aea42ff2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:28:44.765629 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1eec9e1-1f49-4030-bfee-e515aea42ff2 service cinder] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:28:44.777508 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1066/2104] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:28:44 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:28:45.874923 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=24 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:28:54.772109 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=35 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:05.956335 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa984c68-6b3b-4b23-befc-2a017e60ec24 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:05.956933 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa984c68-6b3b-4b23-befc-2a017e60ec24 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:05.957367 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa984c68-6b3b-4b23-befc-2a017e60ec24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:05.978022 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-aa984c68-6b3b-4b23-befc-2a017e60ec24 None None] MFA Rules not processed for user `0696e6f707274727b57a9a7493c47664`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:29:05.979290 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-aa984c68-6b3b-4b23-befc-2a017e60ec24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:29:06.055092 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1039/2105] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:29:05 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:29:11.869624 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1eec9e1-1f49-4030-bfee-e515aea42ff2 service cinder] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:11.871377 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-b1eec9e1-1f49-4030-bfee-e515aea42ff2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:29:11.889595 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa984c68-6b3b-4b23-befc-2a017e60ec24 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:11.893523 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-aa984c68-6b3b-4b23-befc-2a017e60ec24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:29:11.974889 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1eec9e1-1f49-4030-bfee-e515aea42ff2 service cinder] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:11.984141 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b932d8af-decd-4ac6-b8f6-58fafaa6771c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03fc77c6107e430f90f7b6f7cfd4ec02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:11.985805 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b932d8af-decd-4ac6-b8f6-58fafaa6771c service placement] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:11.985805 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b932d8af-decd-4ac6-b8f6-58fafaa6771c service placement] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:11.986122 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b932d8af-decd-4ac6-b8f6-58fafaa6771c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:11.989110 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-b932d8af-decd-4ac6-b8f6-58fafaa6771c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:29:12.003909 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b932d8af-decd-4ac6-b8f6-58fafaa6771c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:12.005711 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b932d8af-decd-4ac6-b8f6-58fafaa6771c service placement] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:12.010969 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa984c68-6b3b-4b23-befc-2a017e60ec24 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:12.018440 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d567dae1-4268-48f2-8f19-27167177afe6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03fc77c6107e430f90f7b6f7cfd4ec02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:12.019159 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d567dae1-4268-48f2-8f19-27167177afe6 service placement] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:12.019602 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d567dae1-4268-48f2-8f19-27167177afe6 service placement] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:12.019830 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1067/2106] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:29:11 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:29:12.023610 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d567dae1-4268-48f2-8f19-27167177afe6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:12.034215 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d567dae1-4268-48f2-8f19-27167177afe6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:12.040727 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d567dae1-4268-48f2-8f19-27167177afe6 service placement] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:12.052675 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1040/2107] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:29:11 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:29:21.882838 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5909ad-8dcd-44f2-b520-19c9cdce21a2 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:21.883399 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5909ad-8dcd-44f2-b520-19c9cdce21a2 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:21.883399 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5909ad-8dcd-44f2-b520-19c9cdce21a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:21.904895 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-0b5909ad-8dcd-44f2-b520-19c9cdce21a2 None None] MFA Rules not processed for user `0ccc6723f7514e52863d102de3839951`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:29:21.906822 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-0b5909ad-8dcd-44f2-b520-19c9cdce21a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:29:22.031257 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1068/2108] 10.222.0.18 () {58 vars in 1009 bytes} [Wed Aug 24 17:29:21 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:29:22.044001 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=57 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:22.135758 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d567dae1-4268-48f2-8f19-27167177afe6 service placement] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:22.137481 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-d567dae1-4268-48f2-8f19-27167177afe6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:29:22.246220 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d567dae1-4268-48f2-8f19-27167177afe6 service placement] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:22.257659 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9faf00-1283-43a7-978d-85ea5e83faf0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef8caa353be34499a58cd25b72ee4032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:22.258752 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9faf00-1283-43a7-978d-85ea5e83faf0 service swift] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:22.263800 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9faf00-1283-43a7-978d-85ea5e83faf0 service swift] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:22.264641 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9faf00-1283-43a7-978d-85ea5e83faf0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:22.280653 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9faf00-1283-43a7-978d-85ea5e83faf0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:22.284051 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9faf00-1283-43a7-978d-85ea5e83faf0 service swift] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:22.294852 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1041/2109] 10.222.0.18 () {60 vars in 1384 bytes} [Wed Aug 24 17:29:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 24 17:29:23.637343 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612b4bce-8165-45ce-b0cf-3d62e08a2cec None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:23.638164 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612b4bce-8165-45ce-b0cf-3d62e08a2cec None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:23.638633 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612b4bce-8165-45ce-b0cf-3d62e08a2cec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:23.665332 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-612b4bce-8165-45ce-b0cf-3d62e08a2cec None None] MFA Rules not processed for user `0ccc6723f7514e52863d102de3839951`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:29:23.666641 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-612b4bce-8165-45ce-b0cf-3d62e08a2cec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:29:23.696581 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1069/2110] 10.222.0.18 () {58 vars in 967 bytes} [Wed Aug 24 17:29:23 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:29:23.706932 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9faf00-1283-43a7-978d-85ea5e83faf0 service swift] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:23.729153 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9faf00-1283-43a7-978d-85ea5e83faf0 service swift] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:23.741160 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172b83ba-0ef6-470a-8230-10d6d0070481 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef8caa353be34499a58cd25b72ee4032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:23.744912 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172b83ba-0ef6-470a-8230-10d6d0070481 service swift] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:23.744912 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172b83ba-0ef6-470a-8230-10d6d0070481 service swift] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:23.744912 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172b83ba-0ef6-470a-8230-10d6d0070481 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:23.773134 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172b83ba-0ef6-470a-8230-10d6d0070481 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:23.776092 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172b83ba-0ef6-470a-8230-10d6d0070481 service swift] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:23.792319 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1042/2111] 10.222.0.18 () {60 vars in 1384 bytes} [Wed Aug 24 17:29:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 17:29:24.446993 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612b4bce-8165-45ce-b0cf-3d62e08a2cec None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:24.568938 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612b4bce-8165-45ce-b0cf-3d62e08a2cec None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:24.579438 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e300a935-60b6-40b5-8e18-90cfbd1a5c31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:24.580235 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e300a935-60b6-40b5-8e18-90cfbd1a5c31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:24.580515 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e300a935-60b6-40b5-8e18-90cfbd1a5c31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:24.580845 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e300a935-60b6-40b5-8e18-90cfbd1a5c31 admin admin] PATH_INFO: `/v3/users/3fa20a7dc3a64475a7c55057a81fb371` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:24.589847 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e300a935-60b6-40b5-8e18-90cfbd1a5c31 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3fa20a7dc3a64475a7c55057a81fb371)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:24.592100 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e300a935-60b6-40b5-8e18-90cfbd1a5c31 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:24.780713 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e36ede-6b19-4c1d-be93-580b368f2d1d None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:24.781292 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e36ede-6b19-4c1d-be93-580b368f2d1d None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:24.781292 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e36ede-6b19-4c1d-be93-580b368f2d1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:24.805476 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-46e36ede-6b19-4c1d-be93-580b368f2d1d None None] MFA Rules not processed for user `0ccc6723f7514e52863d102de3839951`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:29:24.807121 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-46e36ede-6b19-4c1d-be93-580b368f2d1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:29:24.815934 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1043/2112] 10.222.0.18 () {58 vars in 1009 bytes} [Wed Aug 24 17:29:24 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:29:24.843512 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e36ede-6b19-4c1d-be93-580b368f2d1d None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:24.869782 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e36ede-6b19-4c1d-be93-580b368f2d1d None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:24.875810 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa17931-62f3-4e02-b95f-d597ad198d1b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef8caa353be34499a58cd25b72ee4032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:24.876556 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa17931-62f3-4e02-b95f-d597ad198d1b service swift] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:24.876843 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa17931-62f3-4e02-b95f-d597ad198d1b service swift] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:24.877117 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa17931-62f3-4e02-b95f-d597ad198d1b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:24.886339 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa17931-62f3-4e02-b95f-d597ad198d1b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:24.888383 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa17931-62f3-4e02-b95f-d597ad198d1b service swift] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:24.898300 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1044/2113] 10.222.0.18 () {60 vars in 1384 bytes} [Wed Aug 24 17:29:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 24 17:29:24.955520 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:24.969862 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-e300a935-60b6-40b5-8e18-90cfbd1a5c31 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3fa20a7dc3a64475a7c55057a81fb371'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:29:24.984000 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-e300a935-60b6-40b5-8e18-90cfbd1a5c31 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3fa20a7dc3a64475a7c55057a81fb371'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:29:25.087709 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-e300a935-60b6-40b5-8e18-90cfbd1a5c31 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3fa20a7dc3a64475a7c55057a81fb371'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:29:25.088473 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-e300a935-60b6-40b5-8e18-90cfbd1a5c31 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3fa20a7dc3a64475a7c55057a81fb371'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:29:25.100116 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-e300a935-60b6-40b5-8e18-90cfbd1a5c31 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3fa20a7dc3a64475a7c55057a81fb371'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:29:25.107572 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1070/2114] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 24 17:29:24 2022] DELETE /identity/v3/users/3fa20a7dc3a64475a7c55057a81fb371 => generated 0 bytes in 661 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:29:25.111986 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742ca2ac-c547-41bc-87e9-9757289f5be7 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:25.112272 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742ca2ac-c547-41bc-87e9-9757289f5be7 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:25.112547 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742ca2ac-c547-41bc-87e9-9757289f5be7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:25.133495 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-742ca2ac-c547-41bc-87e9-9757289f5be7 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:29:25.134983 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-742ca2ac-c547-41bc-87e9-9757289f5be7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:29:25.212785 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1045/2115] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:29:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:29:25.223352 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e300a935-60b6-40b5-8e18-90cfbd1a5c31 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:25.224839 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-e300a935-60b6-40b5-8e18-90cfbd1a5c31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:29:25.316328 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e300a935-60b6-40b5-8e18-90cfbd1a5c31 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:25.322932 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e304467-a482-44c4-8a7b-8231f3e18418 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e9ed4d47aef4140bfb2d3bb798cc559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:25.327133 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e304467-a482-44c4-8a7b-8231f3e18418 service neutron] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:25.327599 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e304467-a482-44c4-8a7b-8231f3e18418 service neutron] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:25.328054 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e304467-a482-44c4-8a7b-8231f3e18418 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:25.344763 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e304467-a482-44c4-8a7b-8231f3e18418 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:25.347357 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e304467-a482-44c4-8a7b-8231f3e18418 service neutron] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:25.365118 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1071/2116] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:29:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:29:25.728714 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37cc8c9-ea91-4442-aff8-b07850a76813 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:25.739885 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37cc8c9-ea91-4442-aff8-b07850a76813 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:25.739885 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37cc8c9-ea91-4442-aff8-b07850a76813 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:25.752675 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-a37cc8c9-ea91-4442-aff8-b07850a76813 None None] MFA Rules not processed for user `0ccc6723f7514e52863d102de3839951`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:29:25.754043 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-a37cc8c9-ea91-4442-aff8-b07850a76813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:29:25.840138 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1046/2117] 10.222.0.18 () {58 vars in 967 bytes} [Wed Aug 24 17:29:25 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:29:25.858134 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e304467-a482-44c4-8a7b-8231f3e18418 service neutron] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:25.858134 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-2e304467-a482-44c4-8a7b-8231f3e18418 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:29:25.963530 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e304467-a482-44c4-8a7b-8231f3e18418 service neutron] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:25.970226 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089bfa5e-b6fc-4e97-b9b5-86016e33187e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef8caa353be34499a58cd25b72ee4032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:25.971230 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089bfa5e-b6fc-4e97-b9b5-86016e33187e service swift] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:25.971514 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089bfa5e-b6fc-4e97-b9b5-86016e33187e service swift] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:25.971788 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089bfa5e-b6fc-4e97-b9b5-86016e33187e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:25.984881 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089bfa5e-b6fc-4e97-b9b5-86016e33187e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:25.988989 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089bfa5e-b6fc-4e97-b9b5-86016e33187e service swift] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:26.007200 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1072/2118] 10.222.0.18 () {60 vars in 1384 bytes} [Wed Aug 24 17:29:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 17:29:26.056772 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37cc8c9-ea91-4442-aff8-b07850a76813 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:26.058259 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-a37cc8c9-ea91-4442-aff8-b07850a76813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:29:26.069266 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:26.076662 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37cc8c9-ea91-4442-aff8-b07850a76813 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:26.083031 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767311d5-ce82-4f81-8ea3-e49934b3d2f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:26.083882 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767311d5-ce82-4f81-8ea3-e49934b3d2f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:26.084236 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767311d5-ce82-4f81-8ea3-e49934b3d2f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:26.084688 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767311d5-ce82-4f81-8ea3-e49934b3d2f1 admin admin] PATH_INFO: `/v3/projects/e58e1abd4faf4854b65a7d284c588562` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:26.087163 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767311d5-ce82-4f81-8ea3-e49934b3d2f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e58e1abd4faf4854b65a7d284c588562)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:26.089435 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767311d5-ce82-4f81-8ea3-e49934b3d2f1 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:26.213494 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-767311d5-ce82-4f81-8ea3-e49934b3d2f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e58e1abd4faf4854b65a7d284c588562', 'request_id': 'req-767311d5-ce82-4f81-8ea3-e49934b3d2f1'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:29:26.297121 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-767311d5-ce82-4f81-8ea3-e49934b3d2f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e58e1abd4faf4854b65a7d284c588562', 'request_id': 'req-767311d5-ce82-4f81-8ea3-e49934b3d2f1'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:29:26.304289 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-767311d5-ce82-4f81-8ea3-e49934b3d2f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e58e1abd4faf4854b65a7d284c588562', 'request_id': 'req-767311d5-ce82-4f81-8ea3-e49934b3d2f1'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:29:26.315403 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-767311d5-ce82-4f81-8ea3-e49934b3d2f1 admin admin] The token cache is being invalidate because project e58e1abd4faf4854b65a7d284c588562 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81732) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:29:26.315673 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-767311d5-ce82-4f81-8ea3-e49934b3d2f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:29:26.316408 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-767311d5-ce82-4f81-8ea3-e49934b3d2f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:29:26.317913 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1047/2119] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 24 17:29:26 2022] DELETE /identity/v3/projects/e58e1abd4faf4854b65a7d284c588562 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:29:26.322881 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089bfa5e-b6fc-4e97-b9b5-86016e33187e service swift] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:26.337706 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089bfa5e-b6fc-4e97-b9b5-86016e33187e service swift] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:26.396007 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:26.426187 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82bd326-df6f-4e07-bd28-3593180182b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:26.426876 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82bd326-df6f-4e07-bd28-3593180182b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:26.427203 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82bd326-df6f-4e07-bd28-3593180182b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:26.427483 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82bd326-df6f-4e07-bd28-3593180182b2 admin admin] PATH_INFO: `/v3/users/629e894226864c40adc39103b81ec802` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:26.428877 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82bd326-df6f-4e07-bd28-3593180182b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=629e894226864c40adc39103b81ec802)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:26.433132 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82bd326-df6f-4e07-bd28-3593180182b2 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:26.679569 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-d82bd326-df6f-4e07-bd28-3593180182b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '629e894226864c40adc39103b81ec802'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:29:26.695298 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-d82bd326-df6f-4e07-bd28-3593180182b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '629e894226864c40adc39103b81ec802'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:29:26.766360 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-d82bd326-df6f-4e07-bd28-3593180182b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '629e894226864c40adc39103b81ec802'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:29:26.767895 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-d82bd326-df6f-4e07-bd28-3593180182b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '629e894226864c40adc39103b81ec802'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:29:26.780081 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-d82bd326-df6f-4e07-bd28-3593180182b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '629e894226864c40adc39103b81ec802'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:29:26.788329 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1073/2120] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 24 17:29:26 2022] DELETE /identity/v3/users/629e894226864c40adc39103b81ec802 => generated 0 bytes in 467 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:29:27.354742 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767311d5-ce82-4f81-8ea3-e49934b3d2f1 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:27.356465 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-767311d5-ce82-4f81-8ea3-e49934b3d2f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:29:27.376526 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767311d5-ce82-4f81-8ea3-e49934b3d2f1 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:27.476950 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9429b8-bb46-4ce0-9e0c-b3e7a5191710 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:27.477719 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9429b8-bb46-4ce0-9e0c-b3e7a5191710 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:27.478059 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9429b8-bb46-4ce0-9e0c-b3e7a5191710 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:27.481363 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9429b8-bb46-4ce0-9e0c-b3e7a5191710 admin admin] PATH_INFO: `/v3/projects/5b07a5c1f36f43658781f8c330dbb9c9` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:27.481363 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9429b8-bb46-4ce0-9e0c-b3e7a5191710 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b07a5c1f36f43658781f8c330dbb9c9)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:27.482277 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9429b8-bb46-4ce0-9e0c-b3e7a5191710 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:27.610781 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-2c9429b8-bb46-4ce0-9e0c-b3e7a5191710 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b07a5c1f36f43658781f8c330dbb9c9', 'request_id': 'req-2c9429b8-bb46-4ce0-9e0c-b3e7a5191710'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:29:27.675292 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-2c9429b8-bb46-4ce0-9e0c-b3e7a5191710 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b07a5c1f36f43658781f8c330dbb9c9', 'request_id': 'req-2c9429b8-bb46-4ce0-9e0c-b3e7a5191710'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:29:27.683465 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-2c9429b8-bb46-4ce0-9e0c-b3e7a5191710 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b07a5c1f36f43658781f8c330dbb9c9', 'request_id': 'req-2c9429b8-bb46-4ce0-9e0c-b3e7a5191710'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:29:27.695474 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-2c9429b8-bb46-4ce0-9e0c-b3e7a5191710 admin admin] The token cache is being invalidate because project 5b07a5c1f36f43658781f8c330dbb9c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81732) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:29:27.695771 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-2c9429b8-bb46-4ce0-9e0c-b3e7a5191710 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:29:27.696459 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-2c9429b8-bb46-4ce0-9e0c-b3e7a5191710 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:29:27.698002 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1048/2121] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 24 17:29:27 2022] DELETE /identity/v3/projects/5b07a5c1f36f43658781f8c330dbb9c9 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:29:27.705575 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42df61ca-22f1-417a-b393-822a6a552506 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:27.706102 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42df61ca-22f1-417a-b393-822a6a552506 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:27.706409 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42df61ca-22f1-417a-b393-822a6a552506 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:27.735817 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-42df61ca-22f1-417a-b393-822a6a552506 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:29:27.737663 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-42df61ca-22f1-417a-b393-822a6a552506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:29:27.816174 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1074/2122] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:29:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:29:27.825133 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e19533e-62b2-4301-9fa8-bb665e13e741 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:27.825405 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e19533e-62b2-4301-9fa8-bb665e13e741 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:27.825674 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e19533e-62b2-4301-9fa8-bb665e13e741 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:27.845697 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-1e19533e-62b2-4301-9fa8-bb665e13e741 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:29:27.846887 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-1e19533e-62b2-4301-9fa8-bb665e13e741 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:29:27.854955 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1049/2123] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:29:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:29:27.859186 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42df61ca-22f1-417a-b393-822a6a552506 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:27.870547 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42df61ca-22f1-417a-b393-822a6a552506 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:27.875739 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b587f09d-9854-4d01-816f-af7950ea3f8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:27.876357 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b587f09d-9854-4d01-816f-af7950ea3f8f admin admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:27.876643 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b587f09d-9854-4d01-816f-af7950ea3f8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:27.876920 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b587f09d-9854-4d01-816f-af7950ea3f8f admin admin] PATH_INFO: `/v3/domains` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:27.877582 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b587f09d-9854-4d01-816f-af7950ea3f8f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:29:27.877891 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b587f09d-9854-4d01-816f-af7950ea3f8f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:27.879592 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b587f09d-9854-4d01-816f-af7950ea3f8f admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:27.892738 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1075/2124] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 24 17:29:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:27.896930 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e19533e-62b2-4301-9fa8-bb665e13e741 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:27.906137 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e19533e-62b2-4301-9fa8-bb665e13e741 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:27.910957 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b51c44-29e5-4ff2-bb65-220b77879a84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:27.911543 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b51c44-29e5-4ff2-bb65-220b77879a84 admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:27.911839 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b51c44-29e5-4ff2-bb65-220b77879a84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:27.912111 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b51c44-29e5-4ff2-bb65-220b77879a84 admin admin] PATH_INFO: `/v3/projects` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:27.912744 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b51c44-29e5-4ff2-bb65-220b77879a84 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:27.914737 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b51c44-29e5-4ff2-bb65-220b77879a84 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:27.920946 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:28.003250 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1050/2125] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 24 17:29:27 2022] POST /identity/v3/projects => generated 389 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:29:28.008890 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b587f09d-9854-4d01-816f-af7950ea3f8f admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:28.019434 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b587f09d-9854-4d01-816f-af7950ea3f8f admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:28.068203 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:28.082602 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae8220c-4910-4242-a81e-56385da5f9ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:28.083262 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae8220c-4910-4242-a81e-56385da5f9ac admin admin] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:28.083538 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae8220c-4910-4242-a81e-56385da5f9ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:28.083898 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae8220c-4910-4242-a81e-56385da5f9ac admin admin] PATH_INFO: `/v3/users` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:28.084568 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae8220c-4910-4242-a81e-56385da5f9ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:28.086488 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae8220c-4910-4242-a81e-56385da5f9ac admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:28.202473 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1076/2126] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 24 17:29:28 2022] POST /identity/v3/users => generated 295 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:29:28.206631 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b51c44-29e5-4ff2-bb65-220b77879a84 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:28.218090 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b51c44-29e5-4ff2-bb65-220b77879a84 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:28.223541 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f087117e-53f9-4619-8c51-0e254933eb24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:28.224332 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f087117e-53f9-4619-8c51-0e254933eb24 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:28.224626 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f087117e-53f9-4619-8c51-0e254933eb24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:28.224901 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f087117e-53f9-4619-8c51-0e254933eb24 admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:28.226093 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f087117e-53f9-4619-8c51-0e254933eb24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:28.227860 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f087117e-53f9-4619-8c51-0e254933eb24 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:28.240023 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1051/2127] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:29:28 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:29:28.245515 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae8220c-4910-4242-a81e-56385da5f9ac admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:28.256513 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae8220c-4910-4242-a81e-56385da5f9ac admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:28.261413 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69fa955-82b3-4d29-9419-bac5e21608b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:28.262047 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69fa955-82b3-4d29-9419-bac5e21608b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:28.262311 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69fa955-82b3-4d29-9419-bac5e21608b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:28.262590 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69fa955-82b3-4d29-9419-bac5e21608b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:28.263283 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69fa955-82b3-4d29-9419-bac5e21608b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:28.264888 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69fa955-82b3-4d29-9419-bac5e21608b5 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:28.276683 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1077/2128] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:29:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:29:28.281403 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f087117e-53f9-4619-8c51-0e254933eb24 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:28.292172 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f087117e-53f9-4619-8c51-0e254933eb24 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:28.296832 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0eb7e78-cd63-4b0e-8b30-10f98ce501b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:28.297496 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eb7e78-cd63-4b0e-8b30-10f98ce501b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:28.297776 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eb7e78-cd63-4b0e-8b30-10f98ce501b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:28.298045 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eb7e78-cd63-4b0e-8b30-10f98ce501b3 admin admin] PATH_INFO: `/v3/projects/edd81e8f6ae7409191118d133ea01780/users/de2e8289068f4facbffeeaaa982ca765/roles/60dc7e16aee746d5a3e8aa27d7d8525c` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:28.314605 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0eb7e78-cd63-4b0e-8b30-10f98ce501b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=edd81e8f6ae7409191118d133ea01780, user_id=de2e8289068f4facbffeeaaa982ca765, role_id=60dc7e16aee746d5a3e8aa27d7d8525c)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:28.317312 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0eb7e78-cd63-4b0e-8b30-10f98ce501b3 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:28.424584 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1052/2129] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 24 17:29:28 2022] PUT /identity/v3/projects/edd81e8f6ae7409191118d133ea01780/users/de2e8289068f4facbffeeaaa982ca765/roles/60dc7e16aee746d5a3e8aa27d7d8525c => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:29:28.435185 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52339180-ec09-416f-aab1-3f6666a97ec4 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:28.435474 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52339180-ec09-416f-aab1-3f6666a97ec4 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:28.435754 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52339180-ec09-416f-aab1-3f6666a97ec4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:28.465446 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-52339180-ec09-416f-aab1-3f6666a97ec4 None None] MFA Rules not processed for user `de2e8289068f4facbffeeaaa982ca765`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:29:28.466583 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-52339180-ec09-416f-aab1-3f6666a97ec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:29:28.541898 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1078/2130] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:29:28 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:29:28.548001 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0eb7e78-cd63-4b0e-8b30-10f98ce501b3 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:28.647790 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0eb7e78-cd63-4b0e-8b30-10f98ce501b3 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:28.652588 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1d5847-efc8-4748-bfc0-0e11e14d7bbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:28.653201 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1d5847-efc8-4748-bfc0-0e11e14d7bbc admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:28.653485 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1d5847-efc8-4748-bfc0-0e11e14d7bbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:28.653759 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1d5847-efc8-4748-bfc0-0e11e14d7bbc admin admin] PATH_INFO: `/v3/projects` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:28.654381 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1d5847-efc8-4748-bfc0-0e11e14d7bbc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:28.656335 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1d5847-efc8-4748-bfc0-0e11e14d7bbc admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:28.781629 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1053/2131] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 24 17:29:28 2022] POST /identity/v3/projects => generated 387 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:29:28.786550 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52339180-ec09-416f-aab1-3f6666a97ec4 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:28.798627 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52339180-ec09-416f-aab1-3f6666a97ec4 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:28.858107 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949ecbe0-d671-4238-8335-095a87a6ae7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:28.858717 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949ecbe0-d671-4238-8335-095a87a6ae7f admin admin] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:28.859030 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949ecbe0-d671-4238-8335-095a87a6ae7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:28.859308 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949ecbe0-d671-4238-8335-095a87a6ae7f admin admin] PATH_INFO: `/v3/users` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:28.859938 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949ecbe0-d671-4238-8335-095a87a6ae7f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:28.861843 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949ecbe0-d671-4238-8335-095a87a6ae7f admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:28.997801 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1079/2132] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 24 17:29:28 2022] POST /identity/v3/users => generated 300 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:29:29.010156 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1d5847-efc8-4748-bfc0-0e11e14d7bbc admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:29.022212 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1d5847-efc8-4748-bfc0-0e11e14d7bbc admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:29.027599 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc250280-f8b4-40fa-b8ff-98a98ae01e09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:29.028225 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc250280-f8b4-40fa-b8ff-98a98ae01e09 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:29.028508 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc250280-f8b4-40fa-b8ff-98a98ae01e09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:29.028781 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc250280-f8b4-40fa-b8ff-98a98ae01e09 admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:29.029437 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc250280-f8b4-40fa-b8ff-98a98ae01e09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:29.031110 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc250280-f8b4-40fa-b8ff-98a98ae01e09 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:29.043197 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1054/2133] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:29:29 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:29:29.047256 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949ecbe0-d671-4238-8335-095a87a6ae7f admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:29.057695 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949ecbe0-d671-4238-8335-095a87a6ae7f admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:29.062423 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b807b41f-69ad-4bad-8380-202847d955ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:29.063136 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b807b41f-69ad-4bad-8380-202847d955ca admin admin] REQUEST_METHOD: `PUT` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:29.063416 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b807b41f-69ad-4bad-8380-202847d955ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:29.063684 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b807b41f-69ad-4bad-8380-202847d955ca admin admin] PATH_INFO: `/v3/projects/6673cb22e04e45aa85742f1380ae19ae/users/7342201ccd6f4db49390b9f096dc0840/roles/c5346bc54abf445f9900edf94264c850` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:29.080168 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b807b41f-69ad-4bad-8380-202847d955ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=6673cb22e04e45aa85742f1380ae19ae, user_id=7342201ccd6f4db49390b9f096dc0840, role_id=c5346bc54abf445f9900edf94264c850)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:29.082824 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b807b41f-69ad-4bad-8380-202847d955ca admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:29.184777 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1080/2134] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 24 17:29:29 2022] PUT /identity/v3/projects/6673cb22e04e45aa85742f1380ae19ae/users/7342201ccd6f4db49390b9f096dc0840/roles/c5346bc54abf445f9900edf94264c850 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:29:29.188803 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc250280-f8b4-40fa-b8ff-98a98ae01e09 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:29.199910 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc250280-f8b4-40fa-b8ff-98a98ae01e09 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:29.260831 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5740d1f8-9282-44e5-adf1-b6de5f8e183c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:29.261340 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5740d1f8-9282-44e5-adf1-b6de5f8e183c admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:29.261619 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5740d1f8-9282-44e5-adf1-b6de5f8e183c admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:29.261895 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5740d1f8-9282-44e5-adf1-b6de5f8e183c admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:29.262588 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5740d1f8-9282-44e5-adf1-b6de5f8e183c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:29.264231 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5740d1f8-9282-44e5-adf1-b6de5f8e183c admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:29.275673 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1055/2135] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:29:29 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:29:29.280231 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b807b41f-69ad-4bad-8380-202847d955ca admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:29.291748 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b807b41f-69ad-4bad-8380-202847d955ca admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:29.296442 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072ac442-9aac-423a-9fd9-d25b6aacd37a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:29.297107 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072ac442-9aac-423a-9fd9-d25b6aacd37a admin admin] REQUEST_METHOD: `PUT` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:29.297386 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072ac442-9aac-423a-9fd9-d25b6aacd37a admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:29.297659 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072ac442-9aac-423a-9fd9-d25b6aacd37a admin admin] PATH_INFO: `/v3/projects/6673cb22e04e45aa85742f1380ae19ae/users/7342201ccd6f4db49390b9f096dc0840/roles/c5346bc54abf445f9900edf94264c850` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:29.299659 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072ac442-9aac-423a-9fd9-d25b6aacd37a admin admin] RBAC: Authorizing `identity:create_grant(project_id=6673cb22e04e45aa85742f1380ae19ae, user_id=7342201ccd6f4db49390b9f096dc0840, role_id=c5346bc54abf445f9900edf94264c850)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:29.302290 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072ac442-9aac-423a-9fd9-d25b6aacd37a admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:29.313400 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1081/2136] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 24 17:29:29 2022] PUT /identity/v3/projects/6673cb22e04e45aa85742f1380ae19ae/users/7342201ccd6f4db49390b9f096dc0840/roles/c5346bc54abf445f9900edf94264c850 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:29:29.325414 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4760386c-08dc-45db-9baf-2b1cffeb2c43 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:29.325846 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4760386c-08dc-45db-9baf-2b1cffeb2c43 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:29.326216 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4760386c-08dc-45db-9baf-2b1cffeb2c43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:29.357339 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-4760386c-08dc-45db-9baf-2b1cffeb2c43 None None] MFA Rules not processed for user `7342201ccd6f4db49390b9f096dc0840`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:29:29.358557 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-4760386c-08dc-45db-9baf-2b1cffeb2c43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:29:29.368204 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:29.437694 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1056/2137] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:29:29 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:29:29.448839 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072ac442-9aac-423a-9fd9-d25b6aacd37a admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:29.450551 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-072ac442-9aac-423a-9fd9-d25b6aacd37a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:29:29.461151 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:29.518816 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072ac442-9aac-423a-9fd9-d25b6aacd37a admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:29.523432 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f379bd8-63ac-4453-8c88-e0c0894729d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be06246d47584b529150ceb98a5cfd95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:29.524050 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f379bd8-63ac-4453-8c88-e0c0894729d5 service cinder] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:29.524325 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f379bd8-63ac-4453-8c88-e0c0894729d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:29.524599 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f379bd8-63ac-4453-8c88-e0c0894729d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:29.535058 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f379bd8-63ac-4453-8c88-e0c0894729d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:29.536503 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f379bd8-63ac-4453-8c88-e0c0894729d5 service cinder] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:29.547397 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1082/2138] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:29:29 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:29:29.702682 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4760386c-08dc-45db-9baf-2b1cffeb2c43 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:29.769690 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4760386c-08dc-45db-9baf-2b1cffeb2c43 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:29.775720 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43840934-da0c-4f27-9466-37c1033eb0ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:29.776488 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43840934-da0c-4f27-9466-37c1033eb0ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:29.776881 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43840934-da0c-4f27-9466-37c1033eb0ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:29.777289 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43840934-da0c-4f27-9466-37c1033eb0ad admin admin] PATH_INFO: `/v3/users/de2e8289068f4facbffeeaaa982ca765` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:29.779117 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43840934-da0c-4f27-9466-37c1033eb0ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=de2e8289068f4facbffeeaaa982ca765)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:29.782389 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43840934-da0c-4f27-9466-37c1033eb0ad admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:29.944901 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-43840934-da0c-4f27-9466-37c1033eb0ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de2e8289068f4facbffeeaaa982ca765'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:29:30.000447 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-43840934-da0c-4f27-9466-37c1033eb0ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de2e8289068f4facbffeeaaa982ca765'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:29:30.001123 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-43840934-da0c-4f27-9466-37c1033eb0ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de2e8289068f4facbffeeaaa982ca765'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:29:30.013373 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-43840934-da0c-4f27-9466-37c1033eb0ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de2e8289068f4facbffeeaaa982ca765'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:29:30.014108 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-43840934-da0c-4f27-9466-37c1033eb0ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de2e8289068f4facbffeeaaa982ca765'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:29:30.040720 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1057/2139] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 24 17:29:29 2022] DELETE /identity/v3/users/de2e8289068f4facbffeeaaa982ca765 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:29:30.045156 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095f2885-28d5-4c57-a8be-88d4b5536d0a None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:30.045658 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095f2885-28d5-4c57-a8be-88d4b5536d0a None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:30.045757 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095f2885-28d5-4c57-a8be-88d4b5536d0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:30.070091 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-095f2885-28d5-4c57-a8be-88d4b5536d0a None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:29:30.071361 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-095f2885-28d5-4c57-a8be-88d4b5536d0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:29:30.130147 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1083/2140] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:29:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:29:30.141388 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43840934-da0c-4f27-9466-37c1033eb0ad admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:30.143520 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-43840934-da0c-4f27-9466-37c1033eb0ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:29:30.174400 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:30.201724 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43840934-da0c-4f27-9466-37c1033eb0ad admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:30.205809 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c94920-29cc-469d-bbcb-5583c9c5ea89 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e9ed4d47aef4140bfb2d3bb798cc559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:30.206447 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c94920-29cc-469d-bbcb-5583c9c5ea89 service neutron] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:30.206726 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c94920-29cc-469d-bbcb-5583c9c5ea89 service neutron] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:30.207028 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c94920-29cc-469d-bbcb-5583c9c5ea89 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:30.219928 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c94920-29cc-469d-bbcb-5583c9c5ea89 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:30.221453 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c94920-29cc-469d-bbcb-5583c9c5ea89 service neutron] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:30.231612 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1058/2141] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:29:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:29:30.706998 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095f2885-28d5-4c57-a8be-88d4b5536d0a None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:30.708474 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-095f2885-28d5-4c57-a8be-88d4b5536d0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:29:30.724450 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095f2885-28d5-4c57-a8be-88d4b5536d0a None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:30.730741 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86261ec7-54c3-4fa1-a951-3d639194e871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:30.731485 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86261ec7-54c3-4fa1-a951-3d639194e871 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:30.731763 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86261ec7-54c3-4fa1-a951-3d639194e871 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:30.732030 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86261ec7-54c3-4fa1-a951-3d639194e871 admin admin] PATH_INFO: `/v3/projects/edd81e8f6ae7409191118d133ea01780` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:30.733211 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86261ec7-54c3-4fa1-a951-3d639194e871 admin admin] RBAC: Authorizing `identity:delete_project(project_id=edd81e8f6ae7409191118d133ea01780)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:30.736480 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86261ec7-54c3-4fa1-a951-3d639194e871 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:31.108300 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-86261ec7-54c3-4fa1-a951-3d639194e871 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'edd81e8f6ae7409191118d133ea01780', 'request_id': 'req-86261ec7-54c3-4fa1-a951-3d639194e871'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:29:31.201555 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-86261ec7-54c3-4fa1-a951-3d639194e871 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'edd81e8f6ae7409191118d133ea01780', 'request_id': 'req-86261ec7-54c3-4fa1-a951-3d639194e871'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:29:31.211804 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-86261ec7-54c3-4fa1-a951-3d639194e871 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'edd81e8f6ae7409191118d133ea01780', 'request_id': 'req-86261ec7-54c3-4fa1-a951-3d639194e871'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:29:31.234067 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-86261ec7-54c3-4fa1-a951-3d639194e871 admin admin] The token cache is being invalidate because project edd81e8f6ae7409191118d133ea01780 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81731) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:29:31.234626 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-86261ec7-54c3-4fa1-a951-3d639194e871 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:29:31.241259 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-86261ec7-54c3-4fa1-a951-3d639194e871 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:29:31.243118 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1084/2142] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 24 17:29:30 2022] DELETE /identity/v3/projects/edd81e8f6ae7409191118d133ea01780 => generated 0 bytes in 537 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:29:31.248207 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c94920-29cc-469d-bbcb-5583c9c5ea89 service neutron] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:31.268145 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c94920-29cc-469d-bbcb-5583c9c5ea89 service neutron] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:31.369366 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09742292-1739-453e-b9bd-38fc09e3570b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:31.370113 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09742292-1739-453e-b9bd-38fc09e3570b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:31.370482 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09742292-1739-453e-b9bd-38fc09e3570b admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:31.371082 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09742292-1739-453e-b9bd-38fc09e3570b admin admin] PATH_INFO: `/v3/users/7342201ccd6f4db49390b9f096dc0840` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:31.372471 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09742292-1739-453e-b9bd-38fc09e3570b admin admin] RBAC: Authorizing `identity:delete_user(user_id=7342201ccd6f4db49390b9f096dc0840)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:31.374939 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09742292-1739-453e-b9bd-38fc09e3570b admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:31.582479 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-09742292-1739-453e-b9bd-38fc09e3570b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7342201ccd6f4db49390b9f096dc0840'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:29:31.675578 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-09742292-1739-453e-b9bd-38fc09e3570b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7342201ccd6f4db49390b9f096dc0840'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:29:31.676522 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-09742292-1739-453e-b9bd-38fc09e3570b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7342201ccd6f4db49390b9f096dc0840'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:29:31.685548 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-09742292-1739-453e-b9bd-38fc09e3570b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7342201ccd6f4db49390b9f096dc0840'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:29:31.686525 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-09742292-1739-453e-b9bd-38fc09e3570b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7342201ccd6f4db49390b9f096dc0840'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:29:31.708781 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1059/2143] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 24 17:29:31 2022] DELETE /identity/v3/users/7342201ccd6f4db49390b9f096dc0840 => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:29:32.201840 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86261ec7-54c3-4fa1-a951-3d639194e871 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:32.201840 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-86261ec7-54c3-4fa1-a951-3d639194e871 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:29:32.210146 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86261ec7-54c3-4fa1-a951-3d639194e871 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:32.245979 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:32.287341 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b948d80-178f-4b97-a9ce-d427d470edb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:32.289144 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b948d80-178f-4b97-a9ce-d427d470edb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:32.289144 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b948d80-178f-4b97-a9ce-d427d470edb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:32.289338 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b948d80-178f-4b97-a9ce-d427d470edb5 admin admin] PATH_INFO: `/v3/users/e40c080ddcf34210bcf1a75215ee1e43` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:32.290384 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b948d80-178f-4b97-a9ce-d427d470edb5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e40c080ddcf34210bcf1a75215ee1e43)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:32.292416 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b948d80-178f-4b97-a9ce-d427d470edb5 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:32.408103 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09742292-1739-453e-b9bd-38fc09e3570b admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:32.408103 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-09742292-1739-453e-b9bd-38fc09e3570b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:29:32.424955 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09742292-1739-453e-b9bd-38fc09e3570b admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:32.433006 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c19f93-99da-4cf7-8b20-9148723645d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:32.433531 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c19f93-99da-4cf7-8b20-9148723645d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:32.440422 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c19f93-99da-4cf7-8b20-9148723645d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:32.448035 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c19f93-99da-4cf7-8b20-9148723645d4 admin admin] PATH_INFO: `/v3/projects/6673cb22e04e45aa85742f1380ae19ae` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:32.452831 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c19f93-99da-4cf7-8b20-9148723645d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6673cb22e04e45aa85742f1380ae19ae)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:32.456174 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c19f93-99da-4cf7-8b20-9148723645d4 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:32.571937 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-8b948d80-178f-4b97-a9ce-d427d470edb5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e40c080ddcf34210bcf1a75215ee1e43'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:29:32.591795 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-8b948d80-178f-4b97-a9ce-d427d470edb5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e40c080ddcf34210bcf1a75215ee1e43'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:29:32.609285 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:32.615513 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-77c19f93-99da-4cf7-8b20-9148723645d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6673cb22e04e45aa85742f1380ae19ae', 'request_id': 'req-77c19f93-99da-4cf7-8b20-9148723645d4'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:29:32.710376 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-8b948d80-178f-4b97-a9ce-d427d470edb5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e40c080ddcf34210bcf1a75215ee1e43'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:29:32.711631 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-8b948d80-178f-4b97-a9ce-d427d470edb5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e40c080ddcf34210bcf1a75215ee1e43'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:29:32.719749 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-8b948d80-178f-4b97-a9ce-d427d470edb5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e40c080ddcf34210bcf1a75215ee1e43'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:29:32.727922 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1085/2144] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 24 17:29:32 2022] DELETE /identity/v3/users/e40c080ddcf34210bcf1a75215ee1e43 => generated 0 bytes in 534 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:29:32.728566 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3cdc65-6f09-48c6-996f-cba9ba8b7871 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:32.729156 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3cdc65-6f09-48c6-996f-cba9ba8b7871 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:32.729921 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3cdc65-6f09-48c6-996f-cba9ba8b7871 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:32.751496 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-fb3cdc65-6f09-48c6-996f-cba9ba8b7871 None None] MFA Rules not processed for user `0ccc6723f7514e52863d102de3839951`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:29:32.752711 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-fb3cdc65-6f09-48c6-996f-cba9ba8b7871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:29:32.783920 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-77c19f93-99da-4cf7-8b20-9148723645d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6673cb22e04e45aa85742f1380ae19ae', 'request_id': 'req-77c19f93-99da-4cf7-8b20-9148723645d4'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:29:32.800085 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-77c19f93-99da-4cf7-8b20-9148723645d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6673cb22e04e45aa85742f1380ae19ae', 'request_id': 'req-77c19f93-99da-4cf7-8b20-9148723645d4'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:29:32.815115 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-77c19f93-99da-4cf7-8b20-9148723645d4 admin admin] The token cache is being invalidate because project 6673cb22e04e45aa85742f1380ae19ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81732) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:29:32.815578 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-77c19f93-99da-4cf7-8b20-9148723645d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:29:32.816169 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-77c19f93-99da-4cf7-8b20-9148723645d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:29:32.818093 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1060/2145] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 24 17:29:32 2022] DELETE /identity/v3/projects/6673cb22e04e45aa85742f1380ae19ae => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:29:32.819949 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8b3234-12ee-4e97-88ff-678875526f65 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:32.820228 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8b3234-12ee-4e97-88ff-678875526f65 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:32.820561 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8b3234-12ee-4e97-88ff-678875526f65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:32.828295 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1086/2146] 10.222.0.18 () {58 vars in 1009 bytes} [Wed Aug 24 17:29:32 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:29:32.839029 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e71012b-62d1-4396-91e3-79ac9f763d03 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:32.839029 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e71012b-62d1-4396-91e3-79ac9f763d03 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:32.839029 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e71012b-62d1-4396-91e3-79ac9f763d03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:32.848257 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-8f8b3234-12ee-4e97-88ff-678875526f65 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:29:32.850027 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-8f8b3234-12ee-4e97-88ff-678875526f65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:29:32.858296 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-3e71012b-62d1-4396-91e3-79ac9f763d03 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:29:32.861217 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-3e71012b-62d1-4396-91e3-79ac9f763d03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:29:32.935883 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1061/2147] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:29:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:29:32.935883 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8b3234-12ee-4e97-88ff-678875526f65 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:32.935883 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-8f8b3234-12ee-4e97-88ff-678875526f65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:29:32.959811 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1087/2148] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:29:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:29:32.960709 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e71012b-62d1-4396-91e3-79ac9f763d03 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:32.968804 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-3e71012b-62d1-4396-91e3-79ac9f763d03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:29:33.026746 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8b3234-12ee-4e97-88ff-678875526f65 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:33.034712 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0de668-c95b-4873-9f82-cac85d2739d2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef8caa353be34499a58cd25b72ee4032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:33.035663 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0de668-c95b-4873-9f82-cac85d2739d2 service swift] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:33.036071 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0de668-c95b-4873-9f82-cac85d2739d2 service swift] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:33.036454 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0de668-c95b-4873-9f82-cac85d2739d2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:33.042250 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e71012b-62d1-4396-91e3-79ac9f763d03 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:33.046592 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba3f33b-db43-44cf-8a7a-f4eb2efcd51a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e9ed4d47aef4140bfb2d3bb798cc559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:33.047248 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba3f33b-db43-44cf-8a7a-f4eb2efcd51a service neutron] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:33.047520 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba3f33b-db43-44cf-8a7a-f4eb2efcd51a service neutron] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:33.047787 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba3f33b-db43-44cf-8a7a-f4eb2efcd51a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:33.049636 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-bba3f33b-db43-44cf-8a7a-f4eb2efcd51a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:29:33.050872 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0de668-c95b-4873-9f82-cac85d2739d2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:33.053977 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0de668-c95b-4873-9f82-cac85d2739d2 service swift] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:33.065517 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1062/2149] 10.222.0.18 () {60 vars in 1384 bytes} [Wed Aug 24 17:29:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 17:29:33.066392 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0de668-c95b-4873-9f82-cac85d2739d2 service swift] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:33.068021 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-3c0de668-c95b-4873-9f82-cac85d2739d2 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:29:33.081750 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0de668-c95b-4873-9f82-cac85d2739d2 service swift] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:33.085742 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ee062c-de71-48b0-bc03-291dadcecbd2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e9ed4d47aef4140bfb2d3bb798cc559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:33.086347 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ee062c-de71-48b0-bc03-291dadcecbd2 service neutron] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:33.086620 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ee062c-de71-48b0-bc03-291dadcecbd2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:33.086889 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ee062c-de71-48b0-bc03-291dadcecbd2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:33.098891 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ee062c-de71-48b0-bc03-291dadcecbd2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:33.100405 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ee062c-de71-48b0-bc03-291dadcecbd2 service neutron] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:33.111029 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1063/2150] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:29:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:29:33.112492 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68cf1f5-eadf-4032-9948-08f34b5f4d02 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:33.112780 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68cf1f5-eadf-4032-9948-08f34b5f4d02 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:33.113053 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68cf1f5-eadf-4032-9948-08f34b5f4d02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:33.133605 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba3f33b-db43-44cf-8a7a-f4eb2efcd51a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:33.135278 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-a68cf1f5-eadf-4032-9948-08f34b5f4d02 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:29:33.136382 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba3f33b-db43-44cf-8a7a-f4eb2efcd51a service neutron] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:33.136526 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-a68cf1f5-eadf-4032-9948-08f34b5f4d02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:29:33.145484 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1064/2151] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:29:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:29:33.149636 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68cf1f5-eadf-4032-9948-08f34b5f4d02 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:33.150801 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1088/2152] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:29:32 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:29:33.159220 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68cf1f5-eadf-4032-9948-08f34b5f4d02 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:33.167288 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbf6a39-6009-4557-aa23-616aa18362a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:33.168287 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbf6a39-6009-4557-aa23-616aa18362a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:33.168707 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbf6a39-6009-4557-aa23-616aa18362a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:33.169143 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbf6a39-6009-4557-aa23-616aa18362a9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:33.169969 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbf6a39-6009-4557-aa23-616aa18362a9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:29:33.170448 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbf6a39-6009-4557-aa23-616aa18362a9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:33.172898 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbf6a39-6009-4557-aa23-616aa18362a9 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:33.188680 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1065/2153] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 24 17:29:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:29:33.191816 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba3f33b-db43-44cf-8a7a-f4eb2efcd51a service neutron] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:33.205191 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba3f33b-db43-44cf-8a7a-f4eb2efcd51a service neutron] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:33.217434 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa19bf5-ef5e-4361-a243-f0c17f8beead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:33.218418 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa19bf5-ef5e-4361-a243-f0c17f8beead admin admin] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:33.218772 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa19bf5-ef5e-4361-a243-f0c17f8beead admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:33.219411 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa19bf5-ef5e-4361-a243-f0c17f8beead admin admin] PATH_INFO: `/v3/projects` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:33.220422 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa19bf5-ef5e-4361-a243-f0c17f8beead admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:33.222502 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa19bf5-ef5e-4361-a243-f0c17f8beead admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:33.477995 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:33.494103 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbf6a39-6009-4557-aa23-616aa18362a9 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:33.499552 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1089/2154] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 24 17:29:33 2022] POST /identity/v3/projects => generated 409 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:29:33.501393 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-bcbf6a39-6009-4557-aa23-616aa18362a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:29:33.529757 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa19bf5-ef5e-4361-a243-f0c17f8beead admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:33.582553 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbf6a39-6009-4557-aa23-616aa18362a9 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:33.705415 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:33.757625 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa19bf5-ef5e-4361-a243-f0c17f8beead admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:33.775369 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ab7364-ef39-4bec-a55f-6fd9a53acea8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:33.777032 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ab7364-ef39-4bec-a55f-6fd9a53acea8 admin admin] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:33.778962 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ab7364-ef39-4bec-a55f-6fd9a53acea8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:33.782310 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ab7364-ef39-4bec-a55f-6fd9a53acea8 admin admin] PATH_INFO: `/v3/users` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:33.784291 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ab7364-ef39-4bec-a55f-6fd9a53acea8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:33.806340 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ab7364-ef39-4bec-a55f-6fd9a53acea8 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:33.818939 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc2514c-a58d-4986-8b33-005e3ed1cc14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:33.819714 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc2514c-a58d-4986-8b33-005e3ed1cc14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:33.820006 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc2514c-a58d-4986-8b33-005e3ed1cc14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:33.820288 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc2514c-a58d-4986-8b33-005e3ed1cc14 admin admin] PATH_INFO: `/v3/projects/de1ad2d3babc4f32af3f30a3dda9dd48` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:33.822931 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc2514c-a58d-4986-8b33-005e3ed1cc14 admin admin] RBAC: Authorizing `identity:delete_project(project_id=de1ad2d3babc4f32af3f30a3dda9dd48)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:33.825624 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc2514c-a58d-4986-8b33-005e3ed1cc14 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:33.985573 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1090/2155] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 24 17:29:33 2022] POST /identity/v3/users => generated 305 bytes in 458 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:29:33.990682 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ab7364-ef39-4bec-a55f-6fd9a53acea8 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:34.003803 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ab7364-ef39-4bec-a55f-6fd9a53acea8 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:34.012837 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee56edc-b5c8-4781-bb9f-9d5eeb88beb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:34.013592 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee56edc-b5c8-4781-bb9f-9d5eeb88beb3 admin admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:34.014016 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee56edc-b5c8-4781-bb9f-9d5eeb88beb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:34.014790 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee56edc-b5c8-4781-bb9f-9d5eeb88beb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:34.015655 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee56edc-b5c8-4781-bb9f-9d5eeb88beb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:34.017687 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee56edc-b5c8-4781-bb9f-9d5eeb88beb3 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:34.029960 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-8fc2514c-a58d-4986-8b33-005e3ed1cc14 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de1ad2d3babc4f32af3f30a3dda9dd48', 'request_id': 'req-8fc2514c-a58d-4986-8b33-005e3ed1cc14'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:29:34.038346 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1091/2156] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:29:33 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:29:34.042778 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee56edc-b5c8-4781-bb9f-9d5eeb88beb3 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:34.057382 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee56edc-b5c8-4781-bb9f-9d5eeb88beb3 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:34.121540 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-8fc2514c-a58d-4986-8b33-005e3ed1cc14 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de1ad2d3babc4f32af3f30a3dda9dd48', 'request_id': 'req-8fc2514c-a58d-4986-8b33-005e3ed1cc14'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:29:34.145014 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-8fc2514c-a58d-4986-8b33-005e3ed1cc14 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de1ad2d3babc4f32af3f30a3dda9dd48', 'request_id': 'req-8fc2514c-a58d-4986-8b33-005e3ed1cc14'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:29:34.183141 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-8fc2514c-a58d-4986-8b33-005e3ed1cc14 admin admin] The token cache is being invalidate because project de1ad2d3babc4f32af3f30a3dda9dd48 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81732) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:29:34.183141 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-8fc2514c-a58d-4986-8b33-005e3ed1cc14 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:29:34.183141 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-8fc2514c-a58d-4986-8b33-005e3ed1cc14 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:29:34.183141 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1066/2157] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 24 17:29:33 2022] DELETE /identity/v3/projects/de1ad2d3babc4f32af3f30a3dda9dd48 => generated 0 bytes in 684 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:29:34.211489 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d606d170-59a9-4e17-bf62-5ae257a42209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:34.212313 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d606d170-59a9-4e17-bf62-5ae257a42209 admin admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:34.213208 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d606d170-59a9-4e17-bf62-5ae257a42209 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:34.213603 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d606d170-59a9-4e17-bf62-5ae257a42209 admin admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:34.214587 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d606d170-59a9-4e17-bf62-5ae257a42209 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:34.219266 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d606d170-59a9-4e17-bf62-5ae257a42209 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:34.248193 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab188636-0612-4f50-8d0e-7e52dc32fad2 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:34.248653 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab188636-0612-4f50-8d0e-7e52dc32fad2 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:34.249067 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab188636-0612-4f50-8d0e-7e52dc32fad2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:34.283336 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1092/2158] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:29:34 2022] GET /identity/v3/roles => generated 1420 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:29:34.289946 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:34.292330 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d606d170-59a9-4e17-bf62-5ae257a42209 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:34.296787 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-d606d170-59a9-4e17-bf62-5ae257a42209 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:29:34.306934 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-ab188636-0612-4f50-8d0e-7e52dc32fad2 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:29:34.308530 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-ab188636-0612-4f50-8d0e-7e52dc32fad2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:29:34.328256 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d606d170-59a9-4e17-bf62-5ae257a42209 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:34.332464 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1067/2159] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:29:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:29:34.341878 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be51b487-ecc6-4b3e-aba8-d2937ba0351d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:34.345302 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be51b487-ecc6-4b3e-aba8-d2937ba0351d admin admin] REQUEST_METHOD: `PUT` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:34.345510 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be51b487-ecc6-4b3e-aba8-d2937ba0351d admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:34.345991 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be51b487-ecc6-4b3e-aba8-d2937ba0351d admin admin] PATH_INFO: `/v3/projects/04a39a210f734923973ff39b5e232841/users/09b846104d5e47c68dd19c492098a375/roles/60dc7e16aee746d5a3e8aa27d7d8525c` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:34.346950 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceb91ce-4fce-4448-8d8e-e93326e34385 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:34.347855 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceb91ce-4fce-4448-8d8e-e93326e34385 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:34.348659 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceb91ce-4fce-4448-8d8e-e93326e34385 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:34.372296 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be51b487-ecc6-4b3e-aba8-d2937ba0351d admin admin] RBAC: Authorizing `identity:create_grant(project_id=04a39a210f734923973ff39b5e232841, user_id=09b846104d5e47c68dd19c492098a375, role_id=60dc7e16aee746d5a3e8aa27d7d8525c)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:34.378712 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be51b487-ecc6-4b3e-aba8-d2937ba0351d admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:34.416846 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-0ceb91ce-4fce-4448-8d8e-e93326e34385 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:29:34.419052 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-0ceb91ce-4fce-4448-8d8e-e93326e34385 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:29:34.437114 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1068/2160] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:29:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:29:34.443322 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ceb91ce-4fce-4448-8d8e-e93326e34385 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:34.460210 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ceb91ce-4fce-4448-8d8e-e93326e34385 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:34.465754 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642f6dad-e43c-46cd-89db-74f1c246c1fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:34.466464 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642f6dad-e43c-46cd-89db-74f1c246c1fc admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:34.466757 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642f6dad-e43c-46cd-89db-74f1c246c1fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:34.467057 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642f6dad-e43c-46cd-89db-74f1c246c1fc admin admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:34.467796 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642f6dad-e43c-46cd-89db-74f1c246c1fc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:29:34.468100 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642f6dad-e43c-46cd-89db-74f1c246c1fc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:34.469881 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642f6dad-e43c-46cd-89db-74f1c246c1fc admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:34.485741 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1069/2161] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 24 17:29:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:29:34.496283 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642f6dad-e43c-46cd-89db-74f1c246c1fc admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:34.514738 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1093/2162] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 24 17:29:34 2022] PUT /identity/v3/projects/04a39a210f734923973ff39b5e232841/users/09b846104d5e47c68dd19c492098a375/roles/60dc7e16aee746d5a3e8aa27d7d8525c => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:29:34.514966 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642f6dad-e43c-46cd-89db-74f1c246c1fc admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:34.516499 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780166c0-9e8c-4fe5-9323-6e7a56450e52 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:34.516787 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780166c0-9e8c-4fe5-9323-6e7a56450e52 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:34.518744 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780166c0-9e8c-4fe5-9323-6e7a56450e52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:34.569324 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-780166c0-9e8c-4fe5-9323-6e7a56450e52 None None] MFA Rules not processed for user `09b846104d5e47c68dd19c492098a375`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:29:34.570954 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-780166c0-9e8c-4fe5-9323-6e7a56450e52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:29:34.607275 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131a6d22-31f8-4b03-a371-c0a1099d4b59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:34.607949 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131a6d22-31f8-4b03-a371-c0a1099d4b59 admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:34.608295 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131a6d22-31f8-4b03-a371-c0a1099d4b59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:34.608759 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131a6d22-31f8-4b03-a371-c0a1099d4b59 admin admin] PATH_INFO: `/v3/projects` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:34.609398 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131a6d22-31f8-4b03-a371-c0a1099d4b59 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:34.614122 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131a6d22-31f8-4b03-a371-c0a1099d4b59 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:34.682198 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1094/2163] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:29:34 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:29:34.689884 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780166c0-9e8c-4fe5-9323-6e7a56450e52 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:34.728569 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780166c0-9e8c-4fe5-9323-6e7a56450e52 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:34.745967 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1070/2164] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 24 17:29:34 2022] POST /identity/v3/projects => generated 383 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:29:34.752416 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131a6d22-31f8-4b03-a371-c0a1099d4b59 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:34.768622 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bde76bf-eb80-453d-96ac-27a5899e5045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:34.771125 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bde76bf-eb80-453d-96ac-27a5899e5045 admin admin] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:34.774356 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bde76bf-eb80-453d-96ac-27a5899e5045 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:34.775079 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bde76bf-eb80-453d-96ac-27a5899e5045 admin admin] PATH_INFO: `/v3/projects` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:34.776095 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bde76bf-eb80-453d-96ac-27a5899e5045 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:34.778202 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bde76bf-eb80-453d-96ac-27a5899e5045 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:34.827645 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:34.900883 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131a6d22-31f8-4b03-a371-c0a1099d4b59 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:34.914874 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36808a9f-9948-48a7-bcb0-660f45b6b135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:34.915654 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36808a9f-9948-48a7-bcb0-660f45b6b135 admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:34.916526 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36808a9f-9948-48a7-bcb0-660f45b6b135 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:34.916900 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36808a9f-9948-48a7-bcb0-660f45b6b135 admin admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:34.917815 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36808a9f-9948-48a7-bcb0-660f45b6b135 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:34.922126 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36808a9f-9948-48a7-bcb0-660f45b6b135 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:34.961548 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1095/2165] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 24 17:29:34 2022] POST /identity/v3/projects => generated 409 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:29:34.969031 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bde76bf-eb80-453d-96ac-27a5899e5045 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:35.010116 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1071/2166] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 24 17:29:34 2022] POST /identity/v3/users => generated 292 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:29:35.012187 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36808a9f-9948-48a7-bcb0-660f45b6b135 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:35.042636 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36808a9f-9948-48a7-bcb0-660f45b6b135 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:35.077500 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:35.121337 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bde76bf-eb80-453d-96ac-27a5899e5045 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:35.138295 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8744eb-ba9a-49bc-b83c-737ef5d0f9c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:35.138295 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8744eb-ba9a-49bc-b83c-737ef5d0f9c2 admin admin] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:35.139961 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8744eb-ba9a-49bc-b83c-737ef5d0f9c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:35.140397 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8744eb-ba9a-49bc-b83c-737ef5d0f9c2 admin admin] PATH_INFO: `/v3/users` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:35.141190 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8744eb-ba9a-49bc-b83c-737ef5d0f9c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:35.145728 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8744eb-ba9a-49bc-b83c-737ef5d0f9c2 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:35.166943 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341316ad-1b9b-4e7a-ab47-31d9061c39f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:35.167725 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341316ad-1b9b-4e7a-ab47-31d9061c39f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:35.168116 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341316ad-1b9b-4e7a-ab47-31d9061c39f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:35.168809 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341316ad-1b9b-4e7a-ab47-31d9061c39f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:35.172056 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341316ad-1b9b-4e7a-ab47-31d9061c39f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:35.172056 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341316ad-1b9b-4e7a-ab47-31d9061c39f6 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:35.190875 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1072/2167] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:29:35 2022] GET /identity/v3/roles => generated 1420 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:29:35.193612 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341316ad-1b9b-4e7a-ab47-31d9061c39f6 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:35.203838 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341316ad-1b9b-4e7a-ab47-31d9061c39f6 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:35.210944 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444f43ab-a0bd-4edd-913c-3045dc9c2cc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:35.211591 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444f43ab-a0bd-4edd-913c-3045dc9c2cc6 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:35.212015 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444f43ab-a0bd-4edd-913c-3045dc9c2cc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:35.212334 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444f43ab-a0bd-4edd-913c-3045dc9c2cc6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:35.212998 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444f43ab-a0bd-4edd-913c-3045dc9c2cc6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:35.214972 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444f43ab-a0bd-4edd-913c-3045dc9c2cc6 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:35.243811 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1073/2168] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:29:35 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:29:35.246039 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444f43ab-a0bd-4edd-913c-3045dc9c2cc6 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:35.261209 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1096/2169] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 24 17:29:34 2022] POST /identity/v3/users => generated 311 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:29:35.265139 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444f43ab-a0bd-4edd-913c-3045dc9c2cc6 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:35.265855 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8744eb-ba9a-49bc-b83c-737ef5d0f9c2 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:35.271301 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d01337f-3d9a-40c0-a7fa-0b7efcef28e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:35.272634 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d01337f-3d9a-40c0-a7fa-0b7efcef28e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:35.273088 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d01337f-3d9a-40c0-a7fa-0b7efcef28e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:35.273575 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d01337f-3d9a-40c0-a7fa-0b7efcef28e7 admin admin] PATH_INFO: `/v3/projects/95cbbaf120e848a0a699fd6391aebcfd/users/78ce397b5ece4c5b81a1bbefcf87df9d/roles/60dc7e16aee746d5a3e8aa27d7d8525c` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:35.290023 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8744eb-ba9a-49bc-b83c-737ef5d0f9c2 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:35.299448 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f911132e-4731-4984-adbb-c1b5f635931f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:35.300598 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f911132e-4731-4984-adbb-c1b5f635931f admin admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:35.301234 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f911132e-4731-4984-adbb-c1b5f635931f admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:35.301890 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f911132e-4731-4984-adbb-c1b5f635931f admin admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:35.303252 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f911132e-4731-4984-adbb-c1b5f635931f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:35.307468 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f911132e-4731-4984-adbb-c1b5f635931f admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:35.331158 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1097/2170] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:29:35 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:29:35.337486 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f911132e-4731-4984-adbb-c1b5f635931f admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:35.350037 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f911132e-4731-4984-adbb-c1b5f635931f admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:35.353595 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d01337f-3d9a-40c0-a7fa-0b7efcef28e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=95cbbaf120e848a0a699fd6391aebcfd, user_id=78ce397b5ece4c5b81a1bbefcf87df9d, role_id=60dc7e16aee746d5a3e8aa27d7d8525c)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:35.357753 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348033f4-aa68-44ef-a9b5-0ce72a07f18d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:35.358982 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348033f4-aa68-44ef-a9b5-0ce72a07f18d admin admin] REQUEST_METHOD: `PUT` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:35.359219 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348033f4-aa68-44ef-a9b5-0ce72a07f18d admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:35.359601 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348033f4-aa68-44ef-a9b5-0ce72a07f18d admin admin] PATH_INFO: `/v3/projects/f2b8106fa0094680a7236a9aef8b9ef6/users/71c6f612ca58459bb485bdd254bbf6f3/roles/c5346bc54abf445f9900edf94264c850` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:35.377491 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d01337f-3d9a-40c0-a7fa-0b7efcef28e7 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:35.380696 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348033f4-aa68-44ef-a9b5-0ce72a07f18d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2b8106fa0094680a7236a9aef8b9ef6, user_id=71c6f612ca58459bb485bdd254bbf6f3, role_id=c5346bc54abf445f9900edf94264c850)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:35.384369 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348033f4-aa68-44ef-a9b5-0ce72a07f18d admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:35.473893 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1098/2171] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 24 17:29:35 2022] PUT /identity/v3/projects/f2b8106fa0094680a7236a9aef8b9ef6/users/71c6f612ca58459bb485bdd254bbf6f3/roles/c5346bc54abf445f9900edf94264c850 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:29:35.476492 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348033f4-aa68-44ef-a9b5-0ce72a07f18d admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:35.513845 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1074/2172] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 24 17:29:35 2022] PUT /identity/v3/projects/95cbbaf120e848a0a699fd6391aebcfd/users/78ce397b5ece4c5b81a1bbefcf87df9d/roles/60dc7e16aee746d5a3e8aa27d7d8525c => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:29:35.522996 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25572635-e6e9-483a-8c15-04afa88400b5 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:35.523435 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25572635-e6e9-483a-8c15-04afa88400b5 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:35.523810 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25572635-e6e9-483a-8c15-04afa88400b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:35.558357 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-25572635-e6e9-483a-8c15-04afa88400b5 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:29:35.560411 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-25572635-e6e9-483a-8c15-04afa88400b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:29:35.671160 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1075/2173] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:29:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:29:35.717351 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25572635-e6e9-483a-8c15-04afa88400b5 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:35.721293 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-25572635-e6e9-483a-8c15-04afa88400b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:29:35.736445 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348033f4-aa68-44ef-a9b5-0ce72a07f18d admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:35.748480 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:35.756102 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dbfb4d-60e6-42e3-9e99-73bcafb947f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:35.758667 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dbfb4d-60e6-42e3-9e99-73bcafb947f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:35.759131 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dbfb4d-60e6-42e3-9e99-73bcafb947f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:35.759540 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dbfb4d-60e6-42e3-9e99-73bcafb947f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:35.761256 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dbfb4d-60e6-42e3-9e99-73bcafb947f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:35.764265 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dbfb4d-60e6-42e3-9e99-73bcafb947f1 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:35.782558 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1099/2174] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:29:35 2022] GET /identity/v3/roles => generated 1420 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:29:35.788589 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dbfb4d-60e6-42e3-9e99-73bcafb947f1 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:35.824452 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dbfb4d-60e6-42e3-9e99-73bcafb947f1 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:35.836909 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25572635-e6e9-483a-8c15-04afa88400b5 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:35.847380 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3177815-b8d5-46fa-8695-8249e68c7025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:35.849618 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb1a4d6-79c1-4b20-8a24-f10955288e09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e9ed4d47aef4140bfb2d3bb798cc559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:35.849845 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3177815-b8d5-46fa-8695-8249e68c7025 admin admin] REQUEST_METHOD: `PUT` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:35.852054 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb1a4d6-79c1-4b20-8a24-f10955288e09 service neutron] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:35.852300 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3177815-b8d5-46fa-8695-8249e68c7025 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:35.853500 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3177815-b8d5-46fa-8695-8249e68c7025 admin admin] PATH_INFO: `/v3/projects/f2b8106fa0094680a7236a9aef8b9ef6/users/71c6f612ca58459bb485bdd254bbf6f3/roles/c5346bc54abf445f9900edf94264c850` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:35.854291 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb1a4d6-79c1-4b20-8a24-f10955288e09 service neutron] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:35.855870 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb1a4d6-79c1-4b20-8a24-f10955288e09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:35.859802 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3177815-b8d5-46fa-8695-8249e68c7025 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2b8106fa0094680a7236a9aef8b9ef6, user_id=71c6f612ca58459bb485bdd254bbf6f3, role_id=c5346bc54abf445f9900edf94264c850)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:35.865842 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3177815-b8d5-46fa-8695-8249e68c7025 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:35.876522 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb1a4d6-79c1-4b20-8a24-f10955288e09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:35.879075 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb1a4d6-79c1-4b20-8a24-f10955288e09 service neutron] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:35.907180 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1100/2175] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 24 17:29:35 2022] PUT /identity/v3/projects/f2b8106fa0094680a7236a9aef8b9ef6/users/71c6f612ca58459bb485bdd254bbf6f3/roles/c5346bc54abf445f9900edf94264c850 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:29:35.912982 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece7ded7-87fe-4a8c-8f5c-6b9d45d85c38 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:35.916787 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece7ded7-87fe-4a8c-8f5c-6b9d45d85c38 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:35.916787 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece7ded7-87fe-4a8c-8f5c-6b9d45d85c38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:35.934295 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1076/2176] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:29:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:29:35.992344 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-ece7ded7-87fe-4a8c-8f5c-6b9d45d85c38 None None] MFA Rules not processed for user `71c6f612ca58459bb485bdd254bbf6f3`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:29:35.993642 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-ece7ded7-87fe-4a8c-8f5c-6b9d45d85c38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:29:36.185216 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1101/2177] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:29:35 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:29:36.203472 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb1a4d6-79c1-4b20-8a24-f10955288e09 service neutron] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:36.204943 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-1bb1a4d6-79c1-4b20-8a24-f10955288e09 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:29:36.283388 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb1a4d6-79c1-4b20-8a24-f10955288e09 service neutron] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:36.354672 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d9ea0f-98d2-4ec5-abb9-cec43a315551 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be06246d47584b529150ceb98a5cfd95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:36.356520 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d9ea0f-98d2-4ec5-abb9-cec43a315551 service cinder] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:36.357007 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d9ea0f-98d2-4ec5-abb9-cec43a315551 service cinder] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:36.357679 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d9ea0f-98d2-4ec5-abb9-cec43a315551 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:36.384416 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d9ea0f-98d2-4ec5-abb9-cec43a315551 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:36.387115 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d9ea0f-98d2-4ec5-abb9-cec43a315551 service cinder] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:36.403614 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1077/2178] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:29:36 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:29:37.114184 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece7ded7-87fe-4a8c-8f5c-6b9d45d85c38 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:37.129665 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece7ded7-87fe-4a8c-8f5c-6b9d45d85c38 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:37.140346 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f89b99-8d3e-4a4e-964a-9dc6cd231d92 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be06246d47584b529150ceb98a5cfd95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:37.140611 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f89b99-8d3e-4a4e-964a-9dc6cd231d92 service cinder] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:37.142642 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f89b99-8d3e-4a4e-964a-9dc6cd231d92 service cinder] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:37.143214 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f89b99-8d3e-4a4e-964a-9dc6cd231d92 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:37.157407 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:37.304002 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f89b99-8d3e-4a4e-964a-9dc6cd231d92 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:37.316855 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f89b99-8d3e-4a4e-964a-9dc6cd231d92 service cinder] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:37.342152 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1102/2179] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:29:37 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:29:45.828448 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c45c8f-8738-481c-bbf5-4b22641d7251 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:45.829369 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c45c8f-8738-481c-bbf5-4b22641d7251 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:45.829369 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c45c8f-8738-481c-bbf5-4b22641d7251 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:45.864552 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-d5c45c8f-8738-481c-bbf5-4b22641d7251 None None] MFA Rules not processed for user `78ce397b5ece4c5b81a1bbefcf87df9d`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:29:45.866989 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-d5c45c8f-8738-481c-bbf5-4b22641d7251 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:29:45.939374 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:45.944286 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1078/2180] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:29:45 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:29:45.950068 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f89b99-8d3e-4a4e-964a-9dc6cd231d92 service cinder] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:46.026120 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f89b99-8d3e-4a4e-964a-9dc6cd231d92 service cinder] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:46.030826 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e464d531-3d32-4916-9471-8767e13e6b4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:46.031504 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e464d531-3d32-4916-9471-8767e13e6b4c admin admin] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:46.031784 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e464d531-3d32-4916-9471-8767e13e6b4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:46.032056 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e464d531-3d32-4916-9471-8767e13e6b4c admin admin] PATH_INFO: `/v3/projects` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:46.032716 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e464d531-3d32-4916-9471-8767e13e6b4c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:46.034635 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e464d531-3d32-4916-9471-8767e13e6b4c admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:46.111308 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1103/2181] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 24 17:29:45 2022] POST /identity/v3/projects => generated 385 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:29:46.115493 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c45c8f-8738-481c-bbf5-4b22641d7251 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:46.127827 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c45c8f-8738-481c-bbf5-4b22641d7251 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:46.189608 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbc2af8-ccc2-4920-a0fc-e6a39f9a6442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:46.190196 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbc2af8-ccc2-4920-a0fc-e6a39f9a6442 admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:46.190475 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbc2af8-ccc2-4920-a0fc-e6a39f9a6442 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:46.190750 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbc2af8-ccc2-4920-a0fc-e6a39f9a6442 admin admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:46.191450 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbc2af8-ccc2-4920-a0fc-e6a39f9a6442 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:46.194117 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbc2af8-ccc2-4920-a0fc-e6a39f9a6442 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:46.321419 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1079/2182] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 24 17:29:46 2022] POST /identity/v3/users => generated 299 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:29:46.325657 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e464d531-3d32-4916-9471-8767e13e6b4c admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:46.337343 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e464d531-3d32-4916-9471-8767e13e6b4c admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:46.342086 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd33206-7e33-4c9b-a5e9-c3fbd43970c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:46.342702 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd33206-7e33-4c9b-a5e9-c3fbd43970c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:46.343009 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd33206-7e33-4c9b-a5e9-c3fbd43970c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:46.343283 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd33206-7e33-4c9b-a5e9-c3fbd43970c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:46.343930 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd33206-7e33-4c9b-a5e9-c3fbd43970c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:46.345868 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd33206-7e33-4c9b-a5e9-c3fbd43970c4 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:46.356866 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1104/2183] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:29:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:29:46.361157 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbc2af8-ccc2-4920-a0fc-e6a39f9a6442 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:46.371956 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbc2af8-ccc2-4920-a0fc-e6a39f9a6442 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:46.377271 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791ac858-c5b3-416c-9201-a95f97dc463b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:46.377943 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791ac858-c5b3-416c-9201-a95f97dc463b admin admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:46.378221 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791ac858-c5b3-416c-9201-a95f97dc463b admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:46.378538 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791ac858-c5b3-416c-9201-a95f97dc463b admin admin] PATH_INFO: `/v3/projects/cd2951b036184d1c9d4a75f4bd31aa7d/users/1bc9ddfcbd3c44aa8a99823fd1b6bf97/roles/c5346bc54abf445f9900edf94264c850` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:46.395191 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791ac858-c5b3-416c-9201-a95f97dc463b admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd2951b036184d1c9d4a75f4bd31aa7d, user_id=1bc9ddfcbd3c44aa8a99823fd1b6bf97, role_id=c5346bc54abf445f9900edf94264c850)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:46.400863 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791ac858-c5b3-416c-9201-a95f97dc463b admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:46.550400 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1080/2184] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 24 17:29:46 2022] PUT /identity/v3/projects/cd2951b036184d1c9d4a75f4bd31aa7d/users/1bc9ddfcbd3c44aa8a99823fd1b6bf97/roles/c5346bc54abf445f9900edf94264c850 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:29:46.556377 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd33206-7e33-4c9b-a5e9-c3fbd43970c4 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:46.566433 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd33206-7e33-4c9b-a5e9-c3fbd43970c4 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:46.627124 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef7e3cb-715f-473b-b101-448333f61c32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:46.627745 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef7e3cb-715f-473b-b101-448333f61c32 admin admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:46.628052 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef7e3cb-715f-473b-b101-448333f61c32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:46.628326 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef7e3cb-715f-473b-b101-448333f61c32 admin admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:46.628971 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef7e3cb-715f-473b-b101-448333f61c32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:46.630600 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef7e3cb-715f-473b-b101-448333f61c32 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:46.641859 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1105/2185] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:29:46 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:29:46.646513 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791ac858-c5b3-416c-9201-a95f97dc463b admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:46.657215 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791ac858-c5b3-416c-9201-a95f97dc463b admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:46.665167 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadc9171-cb84-4400-93e4-2759e81a7a2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:46.665918 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadc9171-cb84-4400-93e4-2759e81a7a2f admin admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:46.666396 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadc9171-cb84-4400-93e4-2759e81a7a2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:46.666729 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadc9171-cb84-4400-93e4-2759e81a7a2f admin admin] PATH_INFO: `/v3/projects/cd2951b036184d1c9d4a75f4bd31aa7d/users/1bc9ddfcbd3c44aa8a99823fd1b6bf97/roles/c5346bc54abf445f9900edf94264c850` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:46.668723 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadc9171-cb84-4400-93e4-2759e81a7a2f admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd2951b036184d1c9d4a75f4bd31aa7d, user_id=1bc9ddfcbd3c44aa8a99823fd1b6bf97, role_id=c5346bc54abf445f9900edf94264c850)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:46.671491 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadc9171-cb84-4400-93e4-2759e81a7a2f admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:46.682847 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1081/2186] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 24 17:29:46 2022] PUT /identity/v3/projects/cd2951b036184d1c9d4a75f4bd31aa7d/users/1bc9ddfcbd3c44aa8a99823fd1b6bf97/roles/c5346bc54abf445f9900edf94264c850 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:29:54.880602 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef7e3cb-715f-473b-b101-448333f61c32 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:54.897566 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef7e3cb-715f-473b-b101-448333f61c32 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:54.941230 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:55.082818 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a755d6-0514-45fe-88c2-a40cec60183b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:55.084908 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a755d6-0514-45fe-88c2-a40cec60183b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:55.087754 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a755d6-0514-45fe-88c2-a40cec60183b admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:55.090103 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a755d6-0514-45fe-88c2-a40cec60183b admin admin] PATH_INFO: `/v3/users/09b846104d5e47c68dd19c492098a375` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:55.107549 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a755d6-0514-45fe-88c2-a40cec60183b admin admin] RBAC: Authorizing `identity:delete_user(user_id=09b846104d5e47c68dd19c492098a375)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:55.118392 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a755d6-0514-45fe-88c2-a40cec60183b admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:55.442783 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-94a755d6-0514-45fe-88c2-a40cec60183b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09b846104d5e47c68dd19c492098a375'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:29:55.466840 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-94a755d6-0514-45fe-88c2-a40cec60183b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09b846104d5e47c68dd19c492098a375'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:29:55.547275 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-94a755d6-0514-45fe-88c2-a40cec60183b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09b846104d5e47c68dd19c492098a375'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:29:55.547275 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-94a755d6-0514-45fe-88c2-a40cec60183b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09b846104d5e47c68dd19c492098a375'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:29:55.593851 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-94a755d6-0514-45fe-88c2-a40cec60183b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09b846104d5e47c68dd19c492098a375'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:29:55.688150 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1106/2187] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 24 17:29:54 2022] DELETE /identity/v3/users/09b846104d5e47c68dd19c492098a375 => generated 0 bytes in 808 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:29:55.694563 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d3b4a6-bdcc-49f3-ba25-f2791164107d None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:55.695633 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d3b4a6-bdcc-49f3-ba25-f2791164107d None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:55.696087 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d3b4a6-bdcc-49f3-ba25-f2791164107d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:55.731683 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-82d3b4a6-bdcc-49f3-ba25-f2791164107d None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:29:55.735056 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-82d3b4a6-bdcc-49f3-ba25-f2791164107d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:29:55.895601 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1082/2188] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:29:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:29:55.907190 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a755d6-0514-45fe-88c2-a40cec60183b admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:55.909423 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-94a755d6-0514-45fe-88c2-a40cec60183b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:29:56.018651 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a755d6-0514-45fe-88c2-a40cec60183b admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:56.035332 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a3b355-21b5-4398-b739-8bacbdf12209 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e9ed4d47aef4140bfb2d3bb798cc559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:56.036253 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a3b355-21b5-4398-b739-8bacbdf12209 service neutron] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:56.037916 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a3b355-21b5-4398-b739-8bacbdf12209 service neutron] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:56.038302 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a3b355-21b5-4398-b739-8bacbdf12209 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:56.061321 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a3b355-21b5-4398-b739-8bacbdf12209 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:56.064377 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a3b355-21b5-4398-b739-8bacbdf12209 service neutron] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:56.089256 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1107/2189] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:29:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:29:57.904994 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d3b4a6-bdcc-49f3-ba25-f2791164107d None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:57.908839 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-82d3b4a6-bdcc-49f3-ba25-f2791164107d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:29:57.963418 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d3b4a6-bdcc-49f3-ba25-f2791164107d None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:57.981509 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d7c5c9-d840-467c-990b-64d47b1ba628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:57.982978 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d7c5c9-d840-467c-990b-64d47b1ba628 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:57.986202 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d7c5c9-d840-467c-990b-64d47b1ba628 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:57.986789 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d7c5c9-d840-467c-990b-64d47b1ba628 admin admin] PATH_INFO: `/v3/projects/04a39a210f734923973ff39b5e232841` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:57.992776 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d7c5c9-d840-467c-990b-64d47b1ba628 admin admin] RBAC: Authorizing `identity:delete_project(project_id=04a39a210f734923973ff39b5e232841)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:57.999804 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d7c5c9-d840-467c-990b-64d47b1ba628 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:58.154491 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24be7c0e-c9d4-4b44-9597-63cf58a3c01c None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:58.155337 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24be7c0e-c9d4-4b44-9597-63cf58a3c01c None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:58.155500 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24be7c0e-c9d4-4b44-9597-63cf58a3c01c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:58.244649 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-24be7c0e-c9d4-4b44-9597-63cf58a3c01c None None] MFA Rules not processed for user `1bc9ddfcbd3c44aa8a99823fd1b6bf97`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:29:58.247430 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=91,keystone:INSERT=5,keystone:DELETE=4 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:58.250325 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-31d7c5c9-d840-467c-990b-64d47b1ba628 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04a39a210f734923973ff39b5e232841', 'request_id': 'req-31d7c5c9-d840-467c-990b-64d47b1ba628'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:29:58.271914 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-24be7c0e-c9d4-4b44-9597-63cf58a3c01c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:29:58.360958 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:58.367835 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-31d7c5c9-d840-467c-990b-64d47b1ba628 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04a39a210f734923973ff39b5e232841', 'request_id': 'req-31d7c5c9-d840-467c-990b-64d47b1ba628'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:29:58.395564 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-31d7c5c9-d840-467c-990b-64d47b1ba628 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04a39a210f734923973ff39b5e232841', 'request_id': 'req-31d7c5c9-d840-467c-990b-64d47b1ba628'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:29:58.445887 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-31d7c5c9-d840-467c-990b-64d47b1ba628 admin admin] The token cache is being invalidate because project 04a39a210f734923973ff39b5e232841 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81732) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:29:58.446518 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-31d7c5c9-d840-467c-990b-64d47b1ba628 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:29:58.447516 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-31d7c5c9-d840-467c-990b-64d47b1ba628 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:29:58.454683 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1083/2190] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 24 17:29:57 2022] DELETE /identity/v3/projects/04a39a210f734923973ff39b5e232841 => generated 0 bytes in 546 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:29:58.455005 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1108/2191] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:29:58 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:29:58.456412 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d7c5c9-d840-467c-990b-64d47b1ba628 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:58.460070 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-31d7c5c9-d840-467c-990b-64d47b1ba628 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:29:58.468321 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24be7c0e-c9d4-4b44-9597-63cf58a3c01c None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:58.470678 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-24be7c0e-c9d4-4b44-9597-63cf58a3c01c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:29:58.632957 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24be7c0e-c9d4-4b44-9597-63cf58a3c01c None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:58.648280 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4675fd9c-6cb9-4562-8dcd-5bcaf2d44c0d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3553e522363046f6828fd61b2a0642ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:58.648280 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4675fd9c-6cb9-4562-8dcd-5bcaf2d44c0d service glance] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:58.648896 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4675fd9c-6cb9-4562-8dcd-5bcaf2d44c0d service glance] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:58.648896 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4675fd9c-6cb9-4562-8dcd-5bcaf2d44c0d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:58.655587 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-4675fd9c-6cb9-4562-8dcd-5bcaf2d44c0d service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:29:58.684359 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d7c5c9-d840-467c-990b-64d47b1ba628 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:58.713515 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b43d494-0327-4080-bb09-92a2996e386a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:58.715307 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b43d494-0327-4080-bb09-92a2996e386a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:58.715955 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b43d494-0327-4080-bb09-92a2996e386a admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:58.717364 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b43d494-0327-4080-bb09-92a2996e386a admin admin] PATH_INFO: `/v3/users/71c6f612ca58459bb485bdd254bbf6f3` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:58.721911 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b43d494-0327-4080-bb09-92a2996e386a admin admin] RBAC: Authorizing `identity:delete_user(user_id=71c6f612ca58459bb485bdd254bbf6f3)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:58.727933 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b43d494-0327-4080-bb09-92a2996e386a admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:58.934445 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4675fd9c-6cb9-4562-8dcd-5bcaf2d44c0d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:58.940793 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4675fd9c-6cb9-4562-8dcd-5bcaf2d44c0d service glance] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:29:58.979104 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1109/2192] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:29:58 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 511 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:29:59.052064 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-2b43d494-0327-4080-bb09-92a2996e386a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '71c6f612ca58459bb485bdd254bbf6f3'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:29:59.152295 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-2b43d494-0327-4080-bb09-92a2996e386a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '71c6f612ca58459bb485bdd254bbf6f3'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:29:59.156519 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-2b43d494-0327-4080-bb09-92a2996e386a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '71c6f612ca58459bb485bdd254bbf6f3'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:29:59.210697 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-2b43d494-0327-4080-bb09-92a2996e386a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '71c6f612ca58459bb485bdd254bbf6f3'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:29:59.213893 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-2b43d494-0327-4080-bb09-92a2996e386a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '71c6f612ca58459bb485bdd254bbf6f3'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:29:59.278396 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1084/2193] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 24 17:29:58 2022] DELETE /identity/v3/users/71c6f612ca58459bb485bdd254bbf6f3 => generated 0 bytes in 823 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:29:59.470069 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4876b20-3c62-4459-b9af-093f3de7cf65 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:59.471727 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4876b20-3c62-4459-b9af-093f3de7cf65 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:59.472598 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4876b20-3c62-4459-b9af-093f3de7cf65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:59.519449 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-f4876b20-3c62-4459-b9af-093f3de7cf65 None None] MFA Rules not processed for user `0ccc6723f7514e52863d102de3839951`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:29:59.521633 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-f4876b20-3c62-4459-b9af-093f3de7cf65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:29:59.669717 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1110/2194] 10.222.0.18 () {58 vars in 1009 bytes} [Wed Aug 24 17:29:59 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:29:59.714573 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b43d494-0327-4080-bb09-92a2996e386a admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:29:59.716959 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-2b43d494-0327-4080-bb09-92a2996e386a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:29:59.901435 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:29:59.926941 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b43d494-0327-4080-bb09-92a2996e386a admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:29:59.952145 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a626de14-7750-45d9-9793-91883c2e6b17 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef8caa353be34499a58cd25b72ee4032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:29:59.954401 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a626de14-7750-45d9-9793-91883c2e6b17 service swift] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:29:59.955202 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a626de14-7750-45d9-9793-91883c2e6b17 service swift] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:29:59.955945 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a626de14-7750-45d9-9793-91883c2e6b17 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:29:59.989984 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a626de14-7750-45d9-9793-91883c2e6b17 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:29:59.994233 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a626de14-7750-45d9-9793-91883c2e6b17 service swift] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:00.042524 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1085/2195] 10.222.0.18 () {60 vars in 1384 bytes} [Wed Aug 24 17:29:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 17:30:00.190793 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4876b20-3c62-4459-b9af-093f3de7cf65 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:00.197215 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-f4876b20-3c62-4459-b9af-093f3de7cf65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:30:00.389412 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:00.625993 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4876b20-3c62-4459-b9af-093f3de7cf65 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:00.640295 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a957c759-0564-4428-b607-cd3b1118baf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:00.643751 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a957c759-0564-4428-b607-cd3b1118baf0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:00.644453 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a957c759-0564-4428-b607-cd3b1118baf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:00.645013 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a957c759-0564-4428-b607-cd3b1118baf0 admin admin] PATH_INFO: `/v3/projects/f2b8106fa0094680a7236a9aef8b9ef6` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:00.647380 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a957c759-0564-4428-b607-cd3b1118baf0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2b8106fa0094680a7236a9aef8b9ef6)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:00.649492 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a957c759-0564-4428-b607-cd3b1118baf0 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:00.911697 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-a957c759-0564-4428-b607-cd3b1118baf0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2b8106fa0094680a7236a9aef8b9ef6', 'request_id': 'req-a957c759-0564-4428-b607-cd3b1118baf0'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:30:01.000566 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-a957c759-0564-4428-b607-cd3b1118baf0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2b8106fa0094680a7236a9aef8b9ef6', 'request_id': 'req-a957c759-0564-4428-b607-cd3b1118baf0'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:30:01.016713 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-a957c759-0564-4428-b607-cd3b1118baf0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2b8106fa0094680a7236a9aef8b9ef6', 'request_id': 'req-a957c759-0564-4428-b607-cd3b1118baf0'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:30:01.046487 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-a957c759-0564-4428-b607-cd3b1118baf0 admin admin] The token cache is being invalidate because project f2b8106fa0094680a7236a9aef8b9ef6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81731) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:30:01.049344 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-a957c759-0564-4428-b607-cd3b1118baf0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:30:01.053690 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-a957c759-0564-4428-b607-cd3b1118baf0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:30:01.058207 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1111/2196] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 24 17:30:00 2022] DELETE /identity/v3/projects/f2b8106fa0094680a7236a9aef8b9ef6 => generated 0 bytes in 869 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:30:01.066096 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0979a5a8-2056-4673-ad01-8c8865e64e1e None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:01.067752 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0979a5a8-2056-4673-ad01-8c8865e64e1e None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:01.068799 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0979a5a8-2056-4673-ad01-8c8865e64e1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:01.133713 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-0979a5a8-2056-4673-ad01-8c8865e64e1e None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:30:01.137696 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-0979a5a8-2056-4673-ad01-8c8865e64e1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:30:01.458950 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1086/2197] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:30:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 394 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:30:01.461570 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c2cf8c-d6d1-4bba-9660-904677cc24a3 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:01.462193 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c2cf8c-d6d1-4bba-9660-904677cc24a3 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:01.462567 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c2cf8c-d6d1-4bba-9660-904677cc24a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:01.563817 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-28c2cf8c-d6d1-4bba-9660-904677cc24a3 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:30:01.565502 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-28c2cf8c-d6d1-4bba-9660-904677cc24a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:30:01.642168 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1112/2198] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:30:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:30:01.648662 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0979a5a8-2056-4673-ad01-8c8865e64e1e None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:01.715371 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0979a5a8-2056-4673-ad01-8c8865e64e1e None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:01.732489 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0bed44-0381-4069-983d-4d088c1ecf4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:01.733203 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0bed44-0381-4069-983d-4d088c1ecf4d admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:01.733477 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0bed44-0381-4069-983d-4d088c1ecf4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:01.733822 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0bed44-0381-4069-983d-4d088c1ecf4d admin admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:01.736742 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0bed44-0381-4069-983d-4d088c1ecf4d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:01.737136 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0bed44-0381-4069-983d-4d088c1ecf4d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:01.761430 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0bed44-0381-4069-983d-4d088c1ecf4d admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:01.812649 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1087/2199] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 24 17:30:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:01.816338 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c2cf8c-d6d1-4bba-9660-904677cc24a3 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:01.855737 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c2cf8c-d6d1-4bba-9660-904677cc24a3 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:01.877446 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c044cfa-b32e-41ef-b684-0ea02dde4aef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:01.882462 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c044cfa-b32e-41ef-b684-0ea02dde4aef admin admin] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:01.887549 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c044cfa-b32e-41ef-b684-0ea02dde4aef admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:01.888148 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c044cfa-b32e-41ef-b684-0ea02dde4aef admin admin] PATH_INFO: `/v3/projects` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:01.894084 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c044cfa-b32e-41ef-b684-0ea02dde4aef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:01.900662 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c044cfa-b32e-41ef-b684-0ea02dde4aef admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:02.081961 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0bed44-0381-4069-983d-4d088c1ecf4d admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:02.087109 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1113/2200] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 24 17:30:01 2022] POST /identity/v3/projects => generated 377 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:30:02.103332 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0bed44-0381-4069-983d-4d088c1ecf4d admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:02.271327 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b062bf5c-4e26-4973-a81a-b09f019e5913 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:02.271763 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b062bf5c-4e26-4973-a81a-b09f019e5913 admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:02.272049 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b062bf5c-4e26-4973-a81a-b09f019e5913 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:02.272331 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b062bf5c-4e26-4973-a81a-b09f019e5913 admin admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:02.273159 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b062bf5c-4e26-4973-a81a-b09f019e5913 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:02.276508 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b062bf5c-4e26-4973-a81a-b09f019e5913 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:02.405871 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1088/2201] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 24 17:30:02 2022] POST /identity/v3/users => generated 289 bytes in 326 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:30:02.410812 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c044cfa-b32e-41ef-b684-0ea02dde4aef admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:02.425925 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c044cfa-b32e-41ef-b684-0ea02dde4aef admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:02.440744 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9452e067-d7d3-4a77-b21a-598b6ae96e36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:02.444482 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9452e067-d7d3-4a77-b21a-598b6ae96e36 admin admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:02.450233 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9452e067-d7d3-4a77-b21a-598b6ae96e36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:02.450233 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9452e067-d7d3-4a77-b21a-598b6ae96e36 admin admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:02.450233 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9452e067-d7d3-4a77-b21a-598b6ae96e36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:02.452613 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9452e067-d7d3-4a77-b21a-598b6ae96e36 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:02.502047 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b062bf5c-4e26-4973-a81a-b09f019e5913 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:02.521787 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1114/2202] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:30:02 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:30:02.528004 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b062bf5c-4e26-4973-a81a-b09f019e5913 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:02.553309 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc85d61-03b7-4412-96cd-a7c110d27fa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:02.554087 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc85d61-03b7-4412-96cd-a7c110d27fa2 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:02.554657 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc85d61-03b7-4412-96cd-a7c110d27fa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:02.555173 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc85d61-03b7-4412-96cd-a7c110d27fa2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:02.556348 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc85d61-03b7-4412-96cd-a7c110d27fa2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:02.559384 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc85d61-03b7-4412-96cd-a7c110d27fa2 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:02.626922 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9452e067-d7d3-4a77-b21a-598b6ae96e36 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:02.646374 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1089/2203] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:30:02 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:30:02.648944 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9452e067-d7d3-4a77-b21a-598b6ae96e36 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:02.670054 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76560407-7663-46f8-9418-84b842463101 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:02.671083 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76560407-7663-46f8-9418-84b842463101 admin admin] REQUEST_METHOD: `PUT` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:02.671784 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76560407-7663-46f8-9418-84b842463101 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:02.672421 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76560407-7663-46f8-9418-84b842463101 admin admin] PATH_INFO: `/v3/projects/bf7dc55fe4b3421598dd5e0b23e6ed45/users/5ca639ed6e134d64a7919b6828f08a3e/roles/60dc7e16aee746d5a3e8aa27d7d8525c` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:02.718938 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76560407-7663-46f8-9418-84b842463101 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf7dc55fe4b3421598dd5e0b23e6ed45, user_id=5ca639ed6e134d64a7919b6828f08a3e, role_id=60dc7e16aee746d5a3e8aa27d7d8525c)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:02.738672 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76560407-7663-46f8-9418-84b842463101 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:02.936687 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1115/2204] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 24 17:30:02 2022] PUT /identity/v3/projects/bf7dc55fe4b3421598dd5e0b23e6ed45/users/5ca639ed6e134d64a7919b6828f08a3e/roles/60dc7e16aee746d5a3e8aa27d7d8525c => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:30:02.942363 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca348b6e-e752-4c5f-822d-ff2655710fbb None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:02.943008 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca348b6e-e752-4c5f-822d-ff2655710fbb None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:02.943559 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca348b6e-e752-4c5f-822d-ff2655710fbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:03.040088 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-ca348b6e-e752-4c5f-822d-ff2655710fbb None None] MFA Rules not processed for user `5ca639ed6e134d64a7919b6828f08a3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:30:03.042852 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-ca348b6e-e752-4c5f-822d-ff2655710fbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:30:03.234245 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:03.315048 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1090/2205] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:30:02 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 370 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:30:08.978361 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76560407-7663-46f8-9418-84b842463101 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:08.980143 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-76560407-7663-46f8-9418-84b842463101 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:30:09.105389 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:09.114808 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76560407-7663-46f8-9418-84b842463101 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:09.120698 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a2bda5-b861-4fff-9230-d794cdc60a0b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3553e522363046f6828fd61b2a0642ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:09.121962 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a2bda5-b861-4fff-9230-d794cdc60a0b service glance] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:09.135811 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a2bda5-b861-4fff-9230-d794cdc60a0b service glance] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:09.136516 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a2bda5-b861-4fff-9230-d794cdc60a0b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:09.167310 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a2bda5-b861-4fff-9230-d794cdc60a0b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:09.168993 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a2bda5-b861-4fff-9230-d794cdc60a0b service glance] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:09.193426 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1116/2206] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:30:08 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:30:11.371148 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399d3833-2405-4983-93cf-2b27a87b1f74 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:11.371948 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399d3833-2405-4983-93cf-2b27a87b1f74 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:11.371948 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399d3833-2405-4983-93cf-2b27a87b1f74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:11.430629 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-399d3833-2405-4983-93cf-2b27a87b1f74 None None] MFA Rules not processed for user `0ccc6723f7514e52863d102de3839951`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:30:11.434430 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-399d3833-2405-4983-93cf-2b27a87b1f74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:30:11.637006 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1091/2207] 10.222.0.18 () {58 vars in 967 bytes} [Wed Aug 24 17:30:11 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:30:11.682598 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a2bda5-b861-4fff-9230-d794cdc60a0b service glance] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:11.686424 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-28a2bda5-b861-4fff-9230-d794cdc60a0b service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:30:11.832974 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a2bda5-b861-4fff-9230-d794cdc60a0b service glance] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:11.843522 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f06736a-80db-48c7-9c93-9b6f6e7dacbf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef8caa353be34499a58cd25b72ee4032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:11.844587 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f06736a-80db-48c7-9c93-9b6f6e7dacbf service swift] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:11.845268 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f06736a-80db-48c7-9c93-9b6f6e7dacbf service swift] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:11.845899 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f06736a-80db-48c7-9c93-9b6f6e7dacbf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:11.867406 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f06736a-80db-48c7-9c93-9b6f6e7dacbf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:11.873332 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f06736a-80db-48c7-9c93-9b6f6e7dacbf service swift] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:11.895976 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1117/2208] 10.222.0.18 () {60 vars in 1384 bytes} [Wed Aug 24 17:30:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 24 17:30:16.928538 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4de71d3-e232-4ff7-904f-b8cfef8f57e0 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:16.929621 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4de71d3-e232-4ff7-904f-b8cfef8f57e0 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:16.929621 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4de71d3-e232-4ff7-904f-b8cfef8f57e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:16.961058 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-b4de71d3-e232-4ff7-904f-b8cfef8f57e0 None None] MFA Rules not processed for user `78ce397b5ece4c5b81a1bbefcf87df9d`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:30:16.962323 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-b4de71d3-e232-4ff7-904f-b8cfef8f57e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:30:17.074200 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1092/2209] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:30:16 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:30:17.847567 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f06736a-80db-48c7-9c93-9b6f6e7dacbf service swift] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:17.848601 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-0f06736a-80db-48c7-9c93-9b6f6e7dacbf service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:30:17.983368 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f06736a-80db-48c7-9c93-9b6f6e7dacbf service swift] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:17.991820 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667dc019-2abb-4866-a433-9c070a0473a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:17.992709 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667dc019-2abb-4866-a433-9c070a0473a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:17.993134 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667dc019-2abb-4866-a433-9c070a0473a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:17.994461 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667dc019-2abb-4866-a433-9c070a0473a8 admin admin] PATH_INFO: `/v3/users/64eaec18a9d94723b541497bb5e1c105` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:17.996628 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667dc019-2abb-4866-a433-9c070a0473a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=64eaec18a9d94723b541497bb5e1c105)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:17.999633 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667dc019-2abb-4866-a433-9c070a0473a8 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:18.291329 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-667dc019-2abb-4866-a433-9c070a0473a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64eaec18a9d94723b541497bb5e1c105'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:30:18.318785 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-667dc019-2abb-4866-a433-9c070a0473a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64eaec18a9d94723b541497bb5e1c105'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:30:18.389008 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-667dc019-2abb-4866-a433-9c070a0473a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64eaec18a9d94723b541497bb5e1c105'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:30:18.389980 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-667dc019-2abb-4866-a433-9c070a0473a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64eaec18a9d94723b541497bb5e1c105'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:30:18.402689 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-667dc019-2abb-4866-a433-9c070a0473a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64eaec18a9d94723b541497bb5e1c105'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:30:18.405620 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:18.412626 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1118/2210] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 24 17:30:17 2022] DELETE /identity/v3/users/64eaec18a9d94723b541497bb5e1c105 => generated 0 bytes in 567 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:30:18.929377 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4de71d3-e232-4ff7-904f-b8cfef8f57e0 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:18.944968 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4de71d3-e232-4ff7-904f-b8cfef8f57e0 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:19.146052 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9ce72c-d0dc-4a9a-b2a3-4adfb8c6de76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:19.147845 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9ce72c-d0dc-4a9a-b2a3-4adfb8c6de76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:19.148343 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9ce72c-d0dc-4a9a-b2a3-4adfb8c6de76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:19.148864 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9ce72c-d0dc-4a9a-b2a3-4adfb8c6de76 admin admin] PATH_INFO: `/v3/projects/cc0fed6b580c40509ff6514dcd82f4dd` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:19.150693 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9ce72c-d0dc-4a9a-b2a3-4adfb8c6de76 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc0fed6b580c40509ff6514dcd82f4dd)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:19.152998 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9ce72c-d0dc-4a9a-b2a3-4adfb8c6de76 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:19.300613 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:19.327537 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-1c9ce72c-d0dc-4a9a-b2a3-4adfb8c6de76 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc0fed6b580c40509ff6514dcd82f4dd', 'request_id': 'req-1c9ce72c-d0dc-4a9a-b2a3-4adfb8c6de76'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:30:19.441719 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-1c9ce72c-d0dc-4a9a-b2a3-4adfb8c6de76 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc0fed6b580c40509ff6514dcd82f4dd', 'request_id': 'req-1c9ce72c-d0dc-4a9a-b2a3-4adfb8c6de76'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:30:19.456134 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-1c9ce72c-d0dc-4a9a-b2a3-4adfb8c6de76 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc0fed6b580c40509ff6514dcd82f4dd', 'request_id': 'req-1c9ce72c-d0dc-4a9a-b2a3-4adfb8c6de76'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:30:19.475310 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-1c9ce72c-d0dc-4a9a-b2a3-4adfb8c6de76 admin admin] The token cache is being invalidate because project cc0fed6b580c40509ff6514dcd82f4dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81732) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:30:19.475801 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-1c9ce72c-d0dc-4a9a-b2a3-4adfb8c6de76 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:30:19.476896 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-1c9ce72c-d0dc-4a9a-b2a3-4adfb8c6de76 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:30:19.479543 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1093/2211] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 24 17:30:18 2022] DELETE /identity/v3/projects/cc0fed6b580c40509ff6514dcd82f4dd => generated 0 bytes in 551 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:30:19.486845 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd3e35b-0b48-4c15-a22a-49a8a4de04cb None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:19.487435 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd3e35b-0b48-4c15-a22a-49a8a4de04cb None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:19.487706 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd3e35b-0b48-4c15-a22a-49a8a4de04cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:19.520993 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-fcd3e35b-0b48-4c15-a22a-49a8a4de04cb None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:30:19.523048 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-fcd3e35b-0b48-4c15-a22a-49a8a4de04cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:30:19.615123 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1119/2212] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:30:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:30:19.626450 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bc6bfe-2d66-4313-8e72-ac838f0a3942 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:19.629466 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bc6bfe-2d66-4313-8e72-ac838f0a3942 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:19.629466 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bc6bfe-2d66-4313-8e72-ac838f0a3942 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:19.662960 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-99bc6bfe-2d66-4313-8e72-ac838f0a3942 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:30:19.668837 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-99bc6bfe-2d66-4313-8e72-ac838f0a3942 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:30:19.688098 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1094/2213] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:30:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:30:19.690801 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd3e35b-0b48-4c15-a22a-49a8a4de04cb None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:19.717079 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd3e35b-0b48-4c15-a22a-49a8a4de04cb None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:19.726110 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3594103c-df66-47e8-a063-46f58c2de144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:19.727333 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3594103c-df66-47e8-a063-46f58c2de144 admin admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:19.727846 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3594103c-df66-47e8-a063-46f58c2de144 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:19.728398 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3594103c-df66-47e8-a063-46f58c2de144 admin admin] PATH_INFO: `/v3/domains` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:19.729551 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3594103c-df66-47e8-a063-46f58c2de144 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:30:19.730115 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3594103c-df66-47e8-a063-46f58c2de144 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:19.732951 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3594103c-df66-47e8-a063-46f58c2de144 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:19.770275 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1120/2214] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 24 17:30:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:30:19.775624 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bc6bfe-2d66-4313-8e72-ac838f0a3942 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:19.793034 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bc6bfe-2d66-4313-8e72-ac838f0a3942 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:19.801799 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8088aa-aaf0-4afc-8f76-1e6d46fd4951 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:19.802851 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8088aa-aaf0-4afc-8f76-1e6d46fd4951 admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:19.803353 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8088aa-aaf0-4afc-8f76-1e6d46fd4951 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:19.803793 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8088aa-aaf0-4afc-8f76-1e6d46fd4951 admin admin] PATH_INFO: `/v3/projects` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:19.804837 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8088aa-aaf0-4afc-8f76-1e6d46fd4951 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:19.808040 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8088aa-aaf0-4afc-8f76-1e6d46fd4951 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:20.066668 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1095/2215] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 24 17:30:19 2022] POST /identity/v3/projects => generated 365 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:30:20.073284 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3594103c-df66-47e8-a063-46f58c2de144 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:20.090833 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3594103c-df66-47e8-a063-46f58c2de144 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:20.206744 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf43a6d-c166-4b75-9331-8d33638af874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:20.207588 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf43a6d-c166-4b75-9331-8d33638af874 admin admin] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:20.207947 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf43a6d-c166-4b75-9331-8d33638af874 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:20.208295 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf43a6d-c166-4b75-9331-8d33638af874 admin admin] PATH_INFO: `/v3/users` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:20.209646 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf43a6d-c166-4b75-9331-8d33638af874 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:20.212910 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf43a6d-c166-4b75-9331-8d33638af874 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:20.343090 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1121/2216] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 24 17:30:20 2022] POST /identity/v3/users => generated 283 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:30:20.347380 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8088aa-aaf0-4afc-8f76-1e6d46fd4951 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:20.371159 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8088aa-aaf0-4afc-8f76-1e6d46fd4951 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:20.378298 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c503b7-58eb-4d08-a08d-823f53e9f3ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:20.379451 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c503b7-58eb-4d08-a08d-823f53e9f3ef admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:20.379952 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c503b7-58eb-4d08-a08d-823f53e9f3ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:20.380382 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c503b7-58eb-4d08-a08d-823f53e9f3ef admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:20.381441 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c503b7-58eb-4d08-a08d-823f53e9f3ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:20.384317 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c503b7-58eb-4d08-a08d-823f53e9f3ef admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:20.404768 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1096/2217] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:30:20 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:30:20.411369 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf43a6d-c166-4b75-9331-8d33638af874 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:20.429178 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf43a6d-c166-4b75-9331-8d33638af874 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:20.436127 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fef3d6-ddc3-4909-81c8-31df550fd344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:20.437259 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fef3d6-ddc3-4909-81c8-31df550fd344 admin admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:20.437642 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fef3d6-ddc3-4909-81c8-31df550fd344 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:20.438061 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fef3d6-ddc3-4909-81c8-31df550fd344 admin admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:20.440340 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fef3d6-ddc3-4909-81c8-31df550fd344 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:20.442953 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fef3d6-ddc3-4909-81c8-31df550fd344 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:20.458334 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1122/2218] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:30:20 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:30:20.463746 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c503b7-58eb-4d08-a08d-823f53e9f3ef admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:20.479717 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c503b7-58eb-4d08-a08d-823f53e9f3ef admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:20.487985 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702a0e4d-bd27-4b7c-81f1-b48f52573855 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:20.487985 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702a0e4d-bd27-4b7c-81f1-b48f52573855 admin admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:20.487985 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702a0e4d-bd27-4b7c-81f1-b48f52573855 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:20.487985 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702a0e4d-bd27-4b7c-81f1-b48f52573855 admin admin] PATH_INFO: `/v3/projects/d7885a297af4430cb73244a052d3b483/users/15270030171e466595404676937e23bf/roles/60dc7e16aee746d5a3e8aa27d7d8525c` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:20.530250 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702a0e4d-bd27-4b7c-81f1-b48f52573855 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7885a297af4430cb73244a052d3b483, user_id=15270030171e466595404676937e23bf, role_id=60dc7e16aee746d5a3e8aa27d7d8525c)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:20.533434 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702a0e4d-bd27-4b7c-81f1-b48f52573855 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:20.631227 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1097/2219] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 24 17:30:20 2022] PUT /identity/v3/projects/d7885a297af4430cb73244a052d3b483/users/15270030171e466595404676937e23bf/roles/60dc7e16aee746d5a3e8aa27d7d8525c => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:30:20.639489 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b10d23-c39a-4c07-bc0e-42c26eb51488 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:20.640080 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b10d23-c39a-4c07-bc0e-42c26eb51488 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:20.640080 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b10d23-c39a-4c07-bc0e-42c26eb51488 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:20.681778 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-71b10d23-c39a-4c07-bc0e-42c26eb51488 None None] MFA Rules not processed for user `15270030171e466595404676937e23bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:30:20.683539 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-71b10d23-c39a-4c07-bc0e-42c26eb51488 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:30:20.789312 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:20.811106 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1123/2220] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:30:20 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:30:20.820175 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702a0e4d-bd27-4b7c-81f1-b48f52573855 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:20.822438 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-702a0e4d-bd27-4b7c-81f1-b48f52573855 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:30:20.908957 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702a0e4d-bd27-4b7c-81f1-b48f52573855 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:20.912963 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73c1a51-72af-46e2-8956-5881cef3ff40 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be06246d47584b529150ceb98a5cfd95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:20.913583 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73c1a51-72af-46e2-8956-5881cef3ff40 service cinder] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:20.913857 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73c1a51-72af-46e2-8956-5881cef3ff40 service cinder] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:20.914125 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73c1a51-72af-46e2-8956-5881cef3ff40 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:20.929582 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73c1a51-72af-46e2-8956-5881cef3ff40 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:20.932438 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73c1a51-72af-46e2-8956-5881cef3ff40 service cinder] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:20.947902 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1098/2221] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:30:20 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:30:23.197114 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f1238f-b9f9-45b2-b5e4-2de2f233867a None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:23.197930 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f1238f-b9f9-45b2-b5e4-2de2f233867a None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:23.198280 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f1238f-b9f9-45b2-b5e4-2de2f233867a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:23.238781 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-31f1238f-b9f9-45b2-b5e4-2de2f233867a None None] MFA Rules not processed for user `0ccc6723f7514e52863d102de3839951`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:30:23.243488 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-31f1238f-b9f9-45b2-b5e4-2de2f233867a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:30:23.332926 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1124/2222] 10.222.0.18 () {58 vars in 1009 bytes} [Wed Aug 24 17:30:23 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:30:23.362574 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73c1a51-72af-46e2-8956-5881cef3ff40 service cinder] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:23.366500 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-f73c1a51-72af-46e2-8956-5881cef3ff40 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:30:23.492808 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:23.503657 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73c1a51-72af-46e2-8956-5881cef3ff40 service cinder] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:23.511520 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff79c3a-7700-461a-bb1d-8f88c7982a96 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef8caa353be34499a58cd25b72ee4032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:23.511520 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff79c3a-7700-461a-bb1d-8f88c7982a96 service swift] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:23.512085 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff79c3a-7700-461a-bb1d-8f88c7982a96 service swift] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:23.513108 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff79c3a-7700-461a-bb1d-8f88c7982a96 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:23.523383 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff79c3a-7700-461a-bb1d-8f88c7982a96 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:23.525021 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff79c3a-7700-461a-bb1d-8f88c7982a96 service swift] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:23.534602 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1099/2223] 10.222.0.18 () {60 vars in 1384 bytes} [Wed Aug 24 17:30:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 24 17:30:33.326094 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=27 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:33.748289 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=5 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:42.424152 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f1238f-b9f9-45b2-b5e4-2de2f233867a None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:42.425625 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-31f1238f-b9f9-45b2-b5e4-2de2f233867a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:30:42.482630 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f1238f-b9f9-45b2-b5e4-2de2f233867a None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:42.486233 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ac8577-58ea-4563-978b-f4867c63bb51 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e9ed4d47aef4140bfb2d3bb798cc559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:42.486845 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ac8577-58ea-4563-978b-f4867c63bb51 service neutron] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:42.487153 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ac8577-58ea-4563-978b-f4867c63bb51 service neutron] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:42.487425 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ac8577-58ea-4563-978b-f4867c63bb51 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:42.489335 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-47ac8577-58ea-4563-978b-f4867c63bb51 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:30:42.558714 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ac8577-58ea-4563-978b-f4867c63bb51 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:42.560193 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ac8577-58ea-4563-978b-f4867c63bb51 service neutron] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:42.571507 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1125/2224] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:30:42 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:30:49.291790 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff79c3a-7700-461a-bb1d-8f88c7982a96 service swift] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:49.301674 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff79c3a-7700-461a-bb1d-8f88c7982a96 service swift] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:49.306158 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e56781a-44ba-4535-8937-889151e324b3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be06246d47584b529150ceb98a5cfd95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:49.306158 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e56781a-44ba-4535-8937-889151e324b3 service cinder] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:49.307142 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e56781a-44ba-4535-8937-889151e324b3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:49.307328 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e56781a-44ba-4535-8937-889151e324b3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:49.313044 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-2e56781a-44ba-4535-8937-889151e324b3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:30:49.328322 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e56781a-44ba-4535-8937-889151e324b3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:49.329820 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e56781a-44ba-4535-8937-889151e324b3 service cinder] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:49.343602 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1100/2225] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:30:49 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:30:52.566134 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=54 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:30:57.306610 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ac8577-58ea-4563-978b-f4867c63bb51 service neutron] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:57.308258 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-47ac8577-58ea-4563-978b-f4867c63bb51 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:30:57.326310 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ac8577-58ea-4563-978b-f4867c63bb51 service neutron] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:57.333061 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818226c7-cc46-4c1f-998c-dd92fb406e90 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557aca9255cb4bbb8af52e796c5c5177', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:57.333699 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818226c7-cc46-4c1f-998c-dd92fb406e90 service nova] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:57.333977 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818226c7-cc46-4c1f-998c-dd92fb406e90 service nova] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:57.334248 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818226c7-cc46-4c1f-998c-dd92fb406e90 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:57.336754 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-818226c7-cc46-4c1f-998c-dd92fb406e90 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:30:57.428674 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818226c7-cc46-4c1f-998c-dd92fb406e90 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:57.430287 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818226c7-cc46-4c1f-998c-dd92fb406e90 service nova] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:57.440796 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1126/2226] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:30:57 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:30:58.254835 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e56781a-44ba-4535-8937-889151e324b3 service cinder] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:58.257446 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-2e56781a-44ba-4535-8937-889151e324b3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:30:58.336977 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e56781a-44ba-4535-8937-889151e324b3 service cinder] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:58.341494 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a691d068-0e14-47f6-aa52-90850a314f3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:58.342241 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a691d068-0e14-47f6-aa52-90850a314f3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:58.342519 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a691d068-0e14-47f6-aa52-90850a314f3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:58.342792 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a691d068-0e14-47f6-aa52-90850a314f3e admin admin] PATH_INFO: `/v3/users/5ca639ed6e134d64a7919b6828f08a3e` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:58.344031 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a691d068-0e14-47f6-aa52-90850a314f3e admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ca639ed6e134d64a7919b6828f08a3e)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:58.346211 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a691d068-0e14-47f6-aa52-90850a314f3e admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:58.853121 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-a691d068-0e14-47f6-aa52-90850a314f3e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ca639ed6e134d64a7919b6828f08a3e'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:30:58.971851 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-a691d068-0e14-47f6-aa52-90850a314f3e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ca639ed6e134d64a7919b6828f08a3e'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:30:58.972506 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-a691d068-0e14-47f6-aa52-90850a314f3e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ca639ed6e134d64a7919b6828f08a3e'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:30:58.984212 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-a691d068-0e14-47f6-aa52-90850a314f3e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ca639ed6e134d64a7919b6828f08a3e'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:30:58.985595 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-a691d068-0e14-47f6-aa52-90850a314f3e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ca639ed6e134d64a7919b6828f08a3e'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:30:59.015913 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1101/2227] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 24 17:30:58 2022] DELETE /identity/v3/users/5ca639ed6e134d64a7919b6828f08a3e => generated 0 bytes in 762 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:30:59.021158 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269de44d-ec2a-48bd-8223-71263e13fe89 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:59.021697 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269de44d-ec2a-48bd-8223-71263e13fe89 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:59.021697 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269de44d-ec2a-48bd-8223-71263e13fe89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:59.046242 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-269de44d-ec2a-48bd-8223-71263e13fe89 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:30:59.049150 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-269de44d-ec2a-48bd-8223-71263e13fe89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:30:59.162801 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1127/2228] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:30:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:30:59.170594 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a691d068-0e14-47f6-aa52-90850a314f3e admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:30:59.172231 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-a691d068-0e14-47f6-aa52-90850a314f3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:30:59.263340 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a691d068-0e14-47f6-aa52-90850a314f3e admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:30:59.267832 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7aab17-8982-4ccb-b324-8b615f7ab67e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e9ed4d47aef4140bfb2d3bb798cc559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:30:59.268591 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7aab17-8982-4ccb-b324-8b615f7ab67e service neutron] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:30:59.269011 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7aab17-8982-4ccb-b324-8b615f7ab67e service neutron] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:30:59.269652 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7aab17-8982-4ccb-b324-8b615f7ab67e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:30:59.281483 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7aab17-8982-4ccb-b324-8b615f7ab67e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:30:59.283655 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7aab17-8982-4ccb-b324-8b615f7ab67e service neutron] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:30:59.296514 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1102/2229] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:30:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:31:00.437059 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269de44d-ec2a-48bd-8223-71263e13fe89 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:00.442380 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-269de44d-ec2a-48bd-8223-71263e13fe89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:31:00.462048 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269de44d-ec2a-48bd-8223-71263e13fe89 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:00.469224 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41c2bb0-63c3-4791-aac2-bebda46905e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:00.470220 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41c2bb0-63c3-4791-aac2-bebda46905e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:00.470521 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41c2bb0-63c3-4791-aac2-bebda46905e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:00.470810 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41c2bb0-63c3-4791-aac2-bebda46905e2 admin admin] PATH_INFO: `/v3/projects/bf7dc55fe4b3421598dd5e0b23e6ed45` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:00.472331 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41c2bb0-63c3-4791-aac2-bebda46905e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf7dc55fe4b3421598dd5e0b23e6ed45)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:00.475538 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41c2bb0-63c3-4791-aac2-bebda46905e2 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:00.694926 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-a41c2bb0-63c3-4791-aac2-bebda46905e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf7dc55fe4b3421598dd5e0b23e6ed45', 'request_id': 'req-a41c2bb0-63c3-4791-aac2-bebda46905e2'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:31:00.907178 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-a41c2bb0-63c3-4791-aac2-bebda46905e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf7dc55fe4b3421598dd5e0b23e6ed45', 'request_id': 'req-a41c2bb0-63c3-4791-aac2-bebda46905e2'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:31:00.930810 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-a41c2bb0-63c3-4791-aac2-bebda46905e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf7dc55fe4b3421598dd5e0b23e6ed45', 'request_id': 'req-a41c2bb0-63c3-4791-aac2-bebda46905e2'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:31:00.960697 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-a41c2bb0-63c3-4791-aac2-bebda46905e2 admin admin] The token cache is being invalidate because project bf7dc55fe4b3421598dd5e0b23e6ed45 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81731) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:31:00.961006 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-a41c2bb0-63c3-4791-aac2-bebda46905e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:31:00.962647 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-a41c2bb0-63c3-4791-aac2-bebda46905e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:31:00.968730 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1128/2230] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 24 17:31:00 2022] DELETE /identity/v3/projects/bf7dc55fe4b3421598dd5e0b23e6ed45 => generated 0 bytes in 532 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:31:04.369908 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7aab17-8982-4ccb-b324-8b615f7ab67e service neutron] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:04.372209 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-3a7aab17-8982-4ccb-b324-8b615f7ab67e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:31:04.391234 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:31:04.502930 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7aab17-8982-4ccb-b324-8b615f7ab67e service neutron] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:04.620722 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03daa7e0-1cfe-49ba-bccb-8b76e8be7a3b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557aca9255cb4bbb8af52e796c5c5177', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:04.621617 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03daa7e0-1cfe-49ba-bccb-8b76e8be7a3b service nova] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:04.622420 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03daa7e0-1cfe-49ba-bccb-8b76e8be7a3b service nova] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:04.622808 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03daa7e0-1cfe-49ba-bccb-8b76e8be7a3b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:04.629419 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-03daa7e0-1cfe-49ba-bccb-8b76e8be7a3b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:31:04.753087 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03daa7e0-1cfe-49ba-bccb-8b76e8be7a3b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:04.754740 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03daa7e0-1cfe-49ba-bccb-8b76e8be7a3b service nova] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:04.790013 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1103/2231] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:31:04 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:31:05.324364 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41c2bb0-63c3-4791-aac2-bebda46905e2 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:05.326758 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-a41c2bb0-63c3-4791-aac2-bebda46905e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:31:05.363633 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:31:05.399833 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41c2bb0-63c3-4791-aac2-bebda46905e2 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:05.403773 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991d250a-e74c-44e0-b7e5-8264c5826236 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be06246d47584b529150ceb98a5cfd95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:05.404392 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991d250a-e74c-44e0-b7e5-8264c5826236 service cinder] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:05.404674 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991d250a-e74c-44e0-b7e5-8264c5826236 service cinder] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:05.405116 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991d250a-e74c-44e0-b7e5-8264c5826236 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:05.406811 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-991d250a-e74c-44e0-b7e5-8264c5826236 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:31:05.504551 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991d250a-e74c-44e0-b7e5-8264c5826236 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:05.507501 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991d250a-e74c-44e0-b7e5-8264c5826236 service cinder] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:05.524589 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1129/2232] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:31:05 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:31:06.105560 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e1a146-d19f-4222-999d-11f16b74fced None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:06.106188 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e1a146-d19f-4222-999d-11f16b74fced None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:06.106363 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e1a146-d19f-4222-999d-11f16b74fced None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:06.143600 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-d3e1a146-d19f-4222-999d-11f16b74fced None None] MFA Rules not processed for user `0ccc6723f7514e52863d102de3839951`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:31:06.145381 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-d3e1a146-d19f-4222-999d-11f16b74fced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:31:06.222573 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:31:06.251544 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1104/2233] 10.222.0.18 () {58 vars in 1009 bytes} [Wed Aug 24 17:31:06 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:31:06.271144 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991d250a-e74c-44e0-b7e5-8264c5826236 service cinder] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:06.273609 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-991d250a-e74c-44e0-b7e5-8264c5826236 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:31:06.414080 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991d250a-e74c-44e0-b7e5-8264c5826236 service cinder] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:06.421063 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bb09b0-b7b7-40d4-a39c-cd955503ca0d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef8caa353be34499a58cd25b72ee4032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:06.424502 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bb09b0-b7b7-40d4-a39c-cd955503ca0d service swift] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:06.424502 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bb09b0-b7b7-40d4-a39c-cd955503ca0d service swift] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:06.424502 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bb09b0-b7b7-40d4-a39c-cd955503ca0d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:06.431770 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bb09b0-b7b7-40d4-a39c-cd955503ca0d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:06.433400 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bb09b0-b7b7-40d4-a39c-cd955503ca0d service swift] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:06.442652 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1130/2234] 10.222.0.18 () {60 vars in 1384 bytes} [Wed Aug 24 17:31:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 17:31:10.882149 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10484a6a-9dca-4be0-b300-4b2e8aca3939 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:10.896443 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10484a6a-9dca-4be0-b300-4b2e8aca3939 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:10.896443 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10484a6a-9dca-4be0-b300-4b2e8aca3939 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:10.902781 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-10484a6a-9dca-4be0-b300-4b2e8aca3939 None None] MFA Rules not processed for user `160ce86c89ea449da346d98d20ce2a9e`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:31:10.904004 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-10484a6a-9dca-4be0-b300-4b2e8aca3939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:31:10.926829 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1105/2235] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:31:10 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:31:16.341122 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bb09b0-b7b7-40d4-a39c-cd955503ca0d service swift] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:16.342536 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-79bb09b0-b7b7-40d4-a39c-cd955503ca0d service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:31:16.469043 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bb09b0-b7b7-40d4-a39c-cd955503ca0d service swift] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:16.479513 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c74b8b9-717f-49a4-945b-96e3fb247206 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e9ed4d47aef4140bfb2d3bb798cc559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:16.482158 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c74b8b9-717f-49a4-945b-96e3fb247206 service neutron] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:16.482670 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c74b8b9-717f-49a4-945b-96e3fb247206 service neutron] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:16.483203 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c74b8b9-717f-49a4-945b-96e3fb247206 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:16.485945 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-0c74b8b9-717f-49a4-945b-96e3fb247206 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:31:16.501916 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c74b8b9-717f-49a4-945b-96e3fb247206 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:16.504566 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c74b8b9-717f-49a4-945b-96e3fb247206 service neutron] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:16.520317 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1131/2236] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:31:16 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:31:17.177026 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10484a6a-9dca-4be0-b300-4b2e8aca3939 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:17.181626 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-10484a6a-9dca-4be0-b300-4b2e8aca3939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:31:17.416089 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10484a6a-9dca-4be0-b300-4b2e8aca3939 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:17.427814 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46550e6-cca3-4394-a653-149b0ec2cbb3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03fc77c6107e430f90f7b6f7cfd4ec02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:17.428956 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46550e6-cca3-4394-a653-149b0ec2cbb3 service placement] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:17.429452 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46550e6-cca3-4394-a653-149b0ec2cbb3 service placement] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:17.430197 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46550e6-cca3-4394-a653-149b0ec2cbb3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:17.434182 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-f46550e6-cca3-4394-a653-149b0ec2cbb3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:31:17.464279 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46550e6-cca3-4394-a653-149b0ec2cbb3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:17.467447 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46550e6-cca3-4394-a653-149b0ec2cbb3 service placement] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:17.493522 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1106/2237] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:31:17 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:31:20.365775 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c74b8b9-717f-49a4-945b-96e3fb247206 service neutron] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:20.367611 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-0c74b8b9-717f-49a4-945b-96e3fb247206 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:31:20.385882 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c74b8b9-717f-49a4-945b-96e3fb247206 service neutron] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:20.392814 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41001a80-46ae-4581-aef0-43d844eea4d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e9ed4d47aef4140bfb2d3bb798cc559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:20.393978 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41001a80-46ae-4581-aef0-43d844eea4d5 service neutron] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:20.394466 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41001a80-46ae-4581-aef0-43d844eea4d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:20.395249 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41001a80-46ae-4581-aef0-43d844eea4d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:20.398962 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-41001a80-46ae-4581-aef0-43d844eea4d5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:31:20.410560 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:31:20.504649 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41001a80-46ae-4581-aef0-43d844eea4d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:20.506079 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41001a80-46ae-4581-aef0-43d844eea4d5 service neutron] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:20.520397 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1132/2238] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:31:20 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:31:21.284254 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46550e6-cca3-4394-a653-149b0ec2cbb3 service placement] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:21.307981 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46550e6-cca3-4394-a653-149b0ec2cbb3 service placement] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:21.317284 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97aced64-0781-4ef6-9efa-05cee93ffc26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e9ed4d47aef4140bfb2d3bb798cc559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:21.319613 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aced64-0781-4ef6-9efa-05cee93ffc26 service neutron] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:21.320453 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aced64-0781-4ef6-9efa-05cee93ffc26 service neutron] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:21.321169 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aced64-0781-4ef6-9efa-05cee93ffc26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:21.324125 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-97aced64-0781-4ef6-9efa-05cee93ffc26 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:31:21.338771 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97aced64-0781-4ef6-9efa-05cee93ffc26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:21.341600 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97aced64-0781-4ef6-9efa-05cee93ffc26 service neutron] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:21.367926 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1107/2239] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:31:21 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:31:24.809879 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41001a80-46ae-4581-aef0-43d844eea4d5 service neutron] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:24.811688 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-41001a80-46ae-4581-aef0-43d844eea4d5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:31:24.841909 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41001a80-46ae-4581-aef0-43d844eea4d5 service neutron] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:24.848646 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a57574d-8e74-4da9-acf4-d7469100aa98 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557aca9255cb4bbb8af52e796c5c5177', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:24.849927 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a57574d-8e74-4da9-acf4-d7469100aa98 service nova] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:24.850842 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a57574d-8e74-4da9-acf4-d7469100aa98 service nova] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:24.851419 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a57574d-8e74-4da9-acf4-d7469100aa98 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:24.854461 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-7a57574d-8e74-4da9-acf4-d7469100aa98 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:31:24.872433 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a57574d-8e74-4da9-acf4-d7469100aa98 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:24.878974 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a57574d-8e74-4da9-acf4-d7469100aa98 service nova] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:24.891224 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1133/2240] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:31:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:31:31.362235 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=46 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:31:32.815926 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97aced64-0781-4ef6-9efa-05cee93ffc26 service neutron] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:32.827282 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97aced64-0781-4ef6-9efa-05cee93ffc26 service neutron] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:32.831158 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f785a89c-98f7-4cc8-addf-a0e1f6388d6a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e9ed4d47aef4140bfb2d3bb798cc559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:32.831772 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f785a89c-98f7-4cc8-addf-a0e1f6388d6a service neutron] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:32.832050 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f785a89c-98f7-4cc8-addf-a0e1f6388d6a service neutron] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:32.832435 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f785a89c-98f7-4cc8-addf-a0e1f6388d6a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:32.834297 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-f785a89c-98f7-4cc8-addf-a0e1f6388d6a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:31:32.845689 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f785a89c-98f7-4cc8-addf-a0e1f6388d6a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:32.847162 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f785a89c-98f7-4cc8-addf-a0e1f6388d6a service neutron] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:32.857678 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1108/2241] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:31:32 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:31:33.454749 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaf5d29-9750-4a3f-82bd-2a6868f25009 None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:33.455687 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaf5d29-9750-4a3f-82bd-2a6868f25009 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:33.455687 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaf5d29-9750-4a3f-82bd-2a6868f25009 None None] PATH_INFO: `/` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:33.456882 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1134/2242] 10.222.0.18 () {54 vars in 921 bytes} [Wed Aug 24 17:31:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:31:33.463298 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c940799f-679d-4bda-ac02-4712f9f85b95 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:33.466002 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c940799f-679d-4bda-ac02-4712f9f85b95 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:33.466002 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c940799f-679d-4bda-ac02-4712f9f85b95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:33.489070 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-c940799f-679d-4bda-ac02-4712f9f85b95 None None] MFA Rules not processed for user `557aca9255cb4bbb8af52e796c5c5177`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:31:33.491767 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-c940799f-679d-4bda-ac02-4712f9f85b95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:31:33.502687 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1109/2243] 10.222.0.18 () {58 vars in 1018 bytes} [Wed Aug 24 17:31:33 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:31:33.519862 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcaf5d29-9750-4a3f-82bd-2a6868f25009 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:33.521860 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-dcaf5d29-9750-4a3f-82bd-2a6868f25009 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:31:33.592160 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcaf5d29-9750-4a3f-82bd-2a6868f25009 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:33.599460 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde2c744-62d5-4bfe-a13b-b268d13ff1ce service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3553e522363046f6828fd61b2a0642ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:33.599460 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde2c744-62d5-4bfe-a13b-b268d13ff1ce service glance] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:33.599460 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde2c744-62d5-4bfe-a13b-b268d13ff1ce service glance] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:33.599460 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde2c744-62d5-4bfe-a13b-b268d13ff1ce service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:33.611830 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde2c744-62d5-4bfe-a13b-b268d13ff1ce service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:33.613422 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde2c744-62d5-4bfe-a13b-b268d13ff1ce service glance] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:33.638488 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1135/2244] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:31:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:31:33.948494 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8d686a-f439-4a03-94c7-0784383cbe79 None None] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:33.949040 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8d686a-f439-4a03-94c7-0784383cbe79 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:33.949134 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8d686a-f439-4a03-94c7-0784383cbe79 None None] PATH_INFO: `/` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:33.950810 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1110/2245] 10.222.0.18 () {54 vars in 921 bytes} [Wed Aug 24 17:31:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:31:33.957288 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f6d4e0-a881-4a1b-9ec3-30da030c3bbd None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:33.957942 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f6d4e0-a881-4a1b-9ec3-30da030c3bbd None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:33.958417 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f6d4e0-a881-4a1b-9ec3-30da030c3bbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:33.992514 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-09f6d4e0-a881-4a1b-9ec3-30da030c3bbd None None] MFA Rules not processed for user `0e9ed4d47aef4140bfb2d3bb798cc559`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:31:33.992514 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-09f6d4e0-a881-4a1b-9ec3-30da030c3bbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:31:34.000430 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1136/2246] 10.222.0.18 () {58 vars in 1018 bytes} [Wed Aug 24 17:31:33 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:31:34.013637 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8d686a-f439-4a03-94c7-0784383cbe79 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:34.026215 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8d686a-f439-4a03-94c7-0784383cbe79 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:34.030426 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa409f7-99fc-4609-a22f-171625867040 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e9ed4d47aef4140bfb2d3bb798cc559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:34.031068 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa409f7-99fc-4609-a22f-171625867040 service neutron] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:34.031370 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa409f7-99fc-4609-a22f-171625867040 service neutron] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:34.031643 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa409f7-99fc-4609-a22f-171625867040 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:34.040129 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa409f7-99fc-4609-a22f-171625867040 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:34.041590 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa409f7-99fc-4609-a22f-171625867040 service neutron] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:34.051975 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1111/2247] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:31:34 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:31:34.153515 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f503faaa-95ea-48ff-8dfe-221bf51f6f82 None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:34.153898 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f503faaa-95ea-48ff-8dfe-221bf51f6f82 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:34.154247 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f503faaa-95ea-48ff-8dfe-221bf51f6f82 None None] PATH_INFO: `/` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:34.155812 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1137/2248] 10.222.0.18 () {54 vars in 921 bytes} [Wed Aug 24 17:31:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:31:34.164142 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4272b5-3809-4ccc-87a3-f69b4146cb1e None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:34.170134 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4272b5-3809-4ccc-87a3-f69b4146cb1e None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:34.170134 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4272b5-3809-4ccc-87a3-f69b4146cb1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:34.187334 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-2c4272b5-3809-4ccc-87a3-f69b4146cb1e None None] MFA Rules not processed for user `557aca9255cb4bbb8af52e796c5c5177`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:31:34.189070 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-2c4272b5-3809-4ccc-87a3-f69b4146cb1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:31:34.202486 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1112/2249] 10.222.0.18 () {58 vars in 1018 bytes} [Wed Aug 24 17:31:34 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:31:34.217353 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f503faaa-95ea-48ff-8dfe-221bf51f6f82 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:34.232648 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f503faaa-95ea-48ff-8dfe-221bf51f6f82 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:34.237599 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48949592-503c-4553-a88f-323a2b69967c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e9ed4d47aef4140bfb2d3bb798cc559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:34.238517 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48949592-503c-4553-a88f-323a2b69967c service neutron] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:34.238991 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48949592-503c-4553-a88f-323a2b69967c service neutron] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:34.239381 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48949592-503c-4553-a88f-323a2b69967c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:34.253784 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48949592-503c-4553-a88f-323a2b69967c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:34.255087 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48949592-503c-4553-a88f-323a2b69967c service neutron] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:34.269703 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1138/2250] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:31:34 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:31:38.696221 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4272b5-3809-4ccc-87a3-f69b4146cb1e None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:38.710169 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4272b5-3809-4ccc-87a3-f69b4146cb1e None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:38.720084 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7266f0-3b64-4a23-aa5e-7f33aaebe65b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557aca9255cb4bbb8af52e796c5c5177', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:38.721368 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7266f0-3b64-4a23-aa5e-7f33aaebe65b service nova] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:38.721946 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7266f0-3b64-4a23-aa5e-7f33aaebe65b service nova] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:38.722608 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7266f0-3b64-4a23-aa5e-7f33aaebe65b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:38.727368 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-cf7266f0-3b64-4a23-aa5e-7f33aaebe65b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:31:38.745224 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7266f0-3b64-4a23-aa5e-7f33aaebe65b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:38.747571 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7266f0-3b64-4a23-aa5e-7f33aaebe65b service nova] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:38.766044 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1113/2251] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:31:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:31:39.453732 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48949592-503c-4553-a88f-323a2b69967c service neutron] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:39.455697 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-48949592-503c-4553-a88f-323a2b69967c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:31:39.568940 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48949592-503c-4553-a88f-323a2b69967c service neutron] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:39.575054 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662c9f1a-5ba0-4f7d-bf01-b2f9a4183dbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:39.576095 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662c9f1a-5ba0-4f7d-bf01-b2f9a4183dbf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:39.576572 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662c9f1a-5ba0-4f7d-bf01-b2f9a4183dbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:39.577044 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662c9f1a-5ba0-4f7d-bf01-b2f9a4183dbf admin admin] PATH_INFO: `/v3/users/15270030171e466595404676937e23bf` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:39.578805 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662c9f1a-5ba0-4f7d-bf01-b2f9a4183dbf admin admin] RBAC: Authorizing `identity:delete_user(user_id=15270030171e466595404676937e23bf)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:39.581529 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662c9f1a-5ba0-4f7d-bf01-b2f9a4183dbf admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:39.601796 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:31:39.888264 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-662c9f1a-5ba0-4f7d-bf01-b2f9a4183dbf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '15270030171e466595404676937e23bf'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:31:39.904228 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-662c9f1a-5ba0-4f7d-bf01-b2f9a4183dbf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '15270030171e466595404676937e23bf'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:31:40.005503 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-662c9f1a-5ba0-4f7d-bf01-b2f9a4183dbf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '15270030171e466595404676937e23bf'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:31:40.006164 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-662c9f1a-5ba0-4f7d-bf01-b2f9a4183dbf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '15270030171e466595404676937e23bf'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:31:40.017258 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-662c9f1a-5ba0-4f7d-bf01-b2f9a4183dbf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '15270030171e466595404676937e23bf'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:31:40.025076 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1139/2252] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 24 17:31:39 2022] DELETE /identity/v3/users/15270030171e466595404676937e23bf => generated 0 bytes in 572 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:31:40.029473 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fd6647-5bfe-4dfa-a05c-60dd67cc40b4 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:40.030003 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fd6647-5bfe-4dfa-a05c-60dd67cc40b4 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:40.030111 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fd6647-5bfe-4dfa-a05c-60dd67cc40b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:40.051520 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-d9fd6647-5bfe-4dfa-a05c-60dd67cc40b4 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:31:40.052835 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-d9fd6647-5bfe-4dfa-a05c-60dd67cc40b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:31:40.125724 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1114/2253] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:31:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:31:40.137754 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662c9f1a-5ba0-4f7d-bf01-b2f9a4183dbf admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:40.139400 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-662c9f1a-5ba0-4f7d-bf01-b2f9a4183dbf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:31:40.208424 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662c9f1a-5ba0-4f7d-bf01-b2f9a4183dbf admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:40.212897 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31757f3d-1536-4e40-885c-9422dd7771c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e9ed4d47aef4140bfb2d3bb798cc559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:40.215200 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31757f3d-1536-4e40-885c-9422dd7771c1 service neutron] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:40.217333 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31757f3d-1536-4e40-885c-9422dd7771c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:40.217795 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31757f3d-1536-4e40-885c-9422dd7771c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:40.233143 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31757f3d-1536-4e40-885c-9422dd7771c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:40.234610 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31757f3d-1536-4e40-885c-9422dd7771c1 service neutron] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:40.245820 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1140/2254] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:31:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:31:40.737793 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fd6647-5bfe-4dfa-a05c-60dd67cc40b4 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:40.739423 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-d9fd6647-5bfe-4dfa-a05c-60dd67cc40b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:31:40.758700 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fd6647-5bfe-4dfa-a05c-60dd67cc40b4 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:40.766663 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ecdecb-7a54-48ce-b580-24839dca0316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:40.767672 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ecdecb-7a54-48ce-b580-24839dca0316 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:40.767999 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ecdecb-7a54-48ce-b580-24839dca0316 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:40.768482 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ecdecb-7a54-48ce-b580-24839dca0316 admin admin] PATH_INFO: `/v3/projects/d7885a297af4430cb73244a052d3b483` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:40.769979 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ecdecb-7a54-48ce-b580-24839dca0316 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d7885a297af4430cb73244a052d3b483)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:40.772635 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ecdecb-7a54-48ce-b580-24839dca0316 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:40.969866 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-f6ecdecb-7a54-48ce-b580-24839dca0316 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7885a297af4430cb73244a052d3b483', 'request_id': 'req-f6ecdecb-7a54-48ce-b580-24839dca0316'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:31:41.057496 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-f6ecdecb-7a54-48ce-b580-24839dca0316 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7885a297af4430cb73244a052d3b483', 'request_id': 'req-f6ecdecb-7a54-48ce-b580-24839dca0316'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:31:41.063615 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30550e03-a1a3-4f71-8229-7fd59a20d83a None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:41.064149 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30550e03-a1a3-4f71-8229-7fd59a20d83a None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:41.064503 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30550e03-a1a3-4f71-8229-7fd59a20d83a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:41.069711 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-f6ecdecb-7a54-48ce-b580-24839dca0316 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7885a297af4430cb73244a052d3b483', 'request_id': 'req-f6ecdecb-7a54-48ce-b580-24839dca0316'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:31:41.090282 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-30550e03-a1a3-4f71-8229-7fd59a20d83a None None] MFA Rules not processed for user `0ccc6723f7514e52863d102de3839951`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:31:41.091569 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-30550e03-a1a3-4f71-8229-7fd59a20d83a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:31:41.096963 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-f6ecdecb-7a54-48ce-b580-24839dca0316 admin admin] The token cache is being invalidate because project d7885a297af4430cb73244a052d3b483 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81732) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:31:41.098710 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-f6ecdecb-7a54-48ce-b580-24839dca0316 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:31:41.101128 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-f6ecdecb-7a54-48ce-b580-24839dca0316 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:31:41.102849 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1115/2255] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 24 17:31:40 2022] DELETE /identity/v3/projects/d7885a297af4430cb73244a052d3b483 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:31:41.107792 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306025e2-50f0-4241-98ae-9cd482793ff2 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:41.108170 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306025e2-50f0-4241-98ae-9cd482793ff2 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:41.108491 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306025e2-50f0-4241-98ae-9cd482793ff2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:41.147994 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-306025e2-50f0-4241-98ae-9cd482793ff2 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:31:41.147994 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-306025e2-50f0-4241-98ae-9cd482793ff2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:31:41.180091 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:31:41.183830 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1141/2256] 10.222.0.18 () {58 vars in 1009 bytes} [Wed Aug 24 17:31:41 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:31:41.207388 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30550e03-a1a3-4f71-8229-7fd59a20d83a None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:41.208821 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-30550e03-a1a3-4f71-8229-7fd59a20d83a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:31:41.247036 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1116/2257] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:31:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:31:41.257018 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4e85a4-065d-49f5-8e46-af7485ce4d75 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:41.263101 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4e85a4-065d-49f5-8e46-af7485ce4d75 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:41.263101 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4e85a4-065d-49f5-8e46-af7485ce4d75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:41.273769 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:31:41.280925 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-4e4e85a4-065d-49f5-8e46-af7485ce4d75 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:31:41.282224 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-4e4e85a4-065d-49f5-8e46-af7485ce4d75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:31:41.291954 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1117/2258] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:31:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:31:41.297568 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4e85a4-065d-49f5-8e46-af7485ce4d75 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:41.308372 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4e85a4-065d-49f5-8e46-af7485ce4d75 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:41.310763 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30550e03-a1a3-4f71-8229-7fd59a20d83a None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:41.313983 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922d0c2b-c6cd-48a3-bdad-200c93b81928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:41.314617 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922d0c2b-c6cd-48a3-bdad-200c93b81928 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:41.315094 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922d0c2b-c6cd-48a3-bdad-200c93b81928 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:41.315193 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922d0c2b-c6cd-48a3-bdad-200c93b81928 admin admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:41.315821 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922d0c2b-c6cd-48a3-bdad-200c93b81928 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:31:41.316173 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922d0c2b-c6cd-48a3-bdad-200c93b81928 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:41.317460 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8269986-1cbe-4179-9a64-7dc3dc7c12aa service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef8caa353be34499a58cd25b72ee4032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:41.317704 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922d0c2b-c6cd-48a3-bdad-200c93b81928 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:41.318120 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8269986-1cbe-4179-9a64-7dc3dc7c12aa service swift] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:41.318489 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8269986-1cbe-4179-9a64-7dc3dc7c12aa service swift] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:41.318693 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8269986-1cbe-4179-9a64-7dc3dc7c12aa service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:41.328781 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8269986-1cbe-4179-9a64-7dc3dc7c12aa service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:41.330574 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8269986-1cbe-4179-9a64-7dc3dc7c12aa service swift] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:41.332163 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1118/2259] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 24 17:31:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:31:41.344023 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922d0c2b-c6cd-48a3-bdad-200c93b81928 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:41.344174 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1142/2260] 10.222.0.18 () {60 vars in 1384 bytes} [Wed Aug 24 17:31:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 24 17:31:41.353448 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922d0c2b-c6cd-48a3-bdad-200c93b81928 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:41.360313 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d092a6-e931-4070-84df-25e148ad4489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:41.360953 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d092a6-e931-4070-84df-25e148ad4489 admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:41.361238 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d092a6-e931-4070-84df-25e148ad4489 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:41.361505 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d092a6-e931-4070-84df-25e148ad4489 admin admin] PATH_INFO: `/v3/projects` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:41.362148 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d092a6-e931-4070-84df-25e148ad4489 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:41.364094 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d092a6-e931-4070-84df-25e148ad4489 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:41.566264 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1119/2261] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 24 17:31:41 2022] POST /identity/v3/projects => generated 399 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:31:41.572553 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8269986-1cbe-4179-9a64-7dc3dc7c12aa service swift] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:41.591342 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8269986-1cbe-4179-9a64-7dc3dc7c12aa service swift] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:41.711123 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a104744-5298-41c4-a9ba-11c0a560a10a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:41.711749 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a104744-5298-41c4-a9ba-11c0a560a10a admin admin] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:41.712032 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a104744-5298-41c4-a9ba-11c0a560a10a admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:41.712306 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a104744-5298-41c4-a9ba-11c0a560a10a admin admin] PATH_INFO: `/v3/users` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:41.712937 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a104744-5298-41c4-a9ba-11c0a560a10a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:41.714842 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a104744-5298-41c4-a9ba-11c0a560a10a admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:41.853206 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1143/2262] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 24 17:31:41 2022] POST /identity/v3/users => generated 300 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:31:41.854433 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d092a6-e931-4070-84df-25e148ad4489 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:41.874299 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d092a6-e931-4070-84df-25e148ad4489 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:41.880338 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5e52af-b09a-44c5-8487-a9e8c7ceb10f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:41.881059 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5e52af-b09a-44c5-8487-a9e8c7ceb10f admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:41.885923 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5e52af-b09a-44c5-8487-a9e8c7ceb10f admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:41.885923 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5e52af-b09a-44c5-8487-a9e8c7ceb10f admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:41.885923 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5e52af-b09a-44c5-8487-a9e8c7ceb10f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:41.886868 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5e52af-b09a-44c5-8487-a9e8c7ceb10f admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:41.899757 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1120/2263] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:31:41 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:31:41.907474 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a104744-5298-41c4-a9ba-11c0a560a10a admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:41.919439 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a104744-5298-41c4-a9ba-11c0a560a10a admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:41.925386 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0fccf5-7456-4b53-aa2f-a2b0957336d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:41.926225 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0fccf5-7456-4b53-aa2f-a2b0957336d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:41.926641 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0fccf5-7456-4b53-aa2f-a2b0957336d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:41.927159 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0fccf5-7456-4b53-aa2f-a2b0957336d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:41.927965 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0fccf5-7456-4b53-aa2f-a2b0957336d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:41.930330 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0fccf5-7456-4b53-aa2f-a2b0957336d3 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:41.945173 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1144/2264] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:31:41 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:31:41.952489 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5e52af-b09a-44c5-8487-a9e8c7ceb10f admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:41.967179 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5e52af-b09a-44c5-8487-a9e8c7ceb10f admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:41.972356 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee15e2c-5b20-4ac2-bf4d-9d3cfd041978 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:41.973023 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee15e2c-5b20-4ac2-bf4d-9d3cfd041978 admin admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:41.973304 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee15e2c-5b20-4ac2-bf4d-9d3cfd041978 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:41.973577 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee15e2c-5b20-4ac2-bf4d-9d3cfd041978 admin admin] PATH_INFO: `/v3/projects/88dcc46eb6634f90b03ec2819becff6a/users/48c464bcec344322a84ef4b3ce3f8d62/roles/60dc7e16aee746d5a3e8aa27d7d8525c` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:41.995711 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee15e2c-5b20-4ac2-bf4d-9d3cfd041978 admin admin] RBAC: Authorizing `identity:create_grant(project_id=88dcc46eb6634f90b03ec2819becff6a, user_id=48c464bcec344322a84ef4b3ce3f8d62, role_id=60dc7e16aee746d5a3e8aa27d7d8525c)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:42.001368 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee15e2c-5b20-4ac2-bf4d-9d3cfd041978 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:42.152521 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1121/2265] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 24 17:31:41 2022] PUT /identity/v3/projects/88dcc46eb6634f90b03ec2819becff6a/users/48c464bcec344322a84ef4b3ce3f8d62/roles/60dc7e16aee746d5a3e8aa27d7d8525c => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:31:42.222734 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8839d132-2661-4b13-9c61-bb0778499e20 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:42.223359 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8839d132-2661-4b13-9c61-bb0778499e20 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:42.223359 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8839d132-2661-4b13-9c61-bb0778499e20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:42.255106 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-8839d132-2661-4b13-9c61-bb0778499e20 None None] MFA Rules not processed for user `48c464bcec344322a84ef4b3ce3f8d62`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:31:42.262073 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-8839d132-2661-4b13-9c61-bb0778499e20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:31:42.413113 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1145/2266] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:31:42 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:31:42.428409 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee15e2c-5b20-4ac2-bf4d-9d3cfd041978 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:42.430590 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-6ee15e2c-5b20-4ac2-bf4d-9d3cfd041978 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:31:42.551565 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee15e2c-5b20-4ac2-bf4d-9d3cfd041978 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:42.557524 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21e2905-82ec-4072-a4d3-309562df6dc3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be06246d47584b529150ceb98a5cfd95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:42.558338 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21e2905-82ec-4072-a4d3-309562df6dc3 service cinder] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:42.559153 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21e2905-82ec-4072-a4d3-309562df6dc3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:42.559604 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21e2905-82ec-4072-a4d3-309562df6dc3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:42.574128 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21e2905-82ec-4072-a4d3-309562df6dc3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:42.576434 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21e2905-82ec-4072-a4d3-309562df6dc3 service cinder] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:42.601312 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1122/2267] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:31:42 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:31:52.408938 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=80,keystone:INSERT=3 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:31:52.583665 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=72,keystone:INSERT=2 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:31:58.129219 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8839d132-2661-4b13-9c61-bb0778499e20 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:58.195880 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8839d132-2661-4b13-9c61-bb0778499e20 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:58.200566 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf9c03d-9b9d-4b1f-a82f-4e1cb25df3f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:58.201226 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf9c03d-9b9d-4b1f-a82f-4e1cb25df3f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:58.201505 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf9c03d-9b9d-4b1f-a82f-4e1cb25df3f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:58.201779 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf9c03d-9b9d-4b1f-a82f-4e1cb25df3f2 admin admin] PATH_INFO: `/v3/users/48c464bcec344322a84ef4b3ce3f8d62` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:58.203081 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf9c03d-9b9d-4b1f-a82f-4e1cb25df3f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=48c464bcec344322a84ef4b3ce3f8d62)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:58.204992 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf9c03d-9b9d-4b1f-a82f-4e1cb25df3f2 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:58.423012 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-eaf9c03d-9b9d-4b1f-a82f-4e1cb25df3f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '48c464bcec344322a84ef4b3ce3f8d62'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:31:58.436088 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-eaf9c03d-9b9d-4b1f-a82f-4e1cb25df3f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '48c464bcec344322a84ef4b3ce3f8d62'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:31:58.554846 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-eaf9c03d-9b9d-4b1f-a82f-4e1cb25df3f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '48c464bcec344322a84ef4b3ce3f8d62'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:31:58.555537 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-eaf9c03d-9b9d-4b1f-a82f-4e1cb25df3f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '48c464bcec344322a84ef4b3ce3f8d62'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:31:58.563058 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-eaf9c03d-9b9d-4b1f-a82f-4e1cb25df3f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '48c464bcec344322a84ef4b3ce3f8d62'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:31:58.570244 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1146/2268] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 24 17:31:58 2022] DELETE /identity/v3/users/48c464bcec344322a84ef4b3ce3f8d62 => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:31:58.575013 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06392e0-5c0b-4bc1-a415-80c127007e07 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:58.575535 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06392e0-5c0b-4bc1-a415-80c127007e07 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:58.575636 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06392e0-5c0b-4bc1-a415-80c127007e07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:58.596287 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-b06392e0-5c0b-4bc1-a415-80c127007e07 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:31:58.597517 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-b06392e0-5c0b-4bc1-a415-80c127007e07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:31:58.664751 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1123/2269] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:31:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:31:58.674821 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf9c03d-9b9d-4b1f-a82f-4e1cb25df3f2 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:58.676207 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-eaf9c03d-9b9d-4b1f-a82f-4e1cb25df3f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:31:58.737062 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf9c03d-9b9d-4b1f-a82f-4e1cb25df3f2 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:58.741415 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6046089-ccaa-49d9-9c64-4d0f21736e19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e9ed4d47aef4140bfb2d3bb798cc559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:58.742021 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6046089-ccaa-49d9-9c64-4d0f21736e19 service neutron] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:58.742303 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6046089-ccaa-49d9-9c64-4d0f21736e19 service neutron] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:58.742577 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6046089-ccaa-49d9-9c64-4d0f21736e19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:58.751859 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6046089-ccaa-49d9-9c64-4d0f21736e19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:58.753297 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6046089-ccaa-49d9-9c64-4d0f21736e19 service neutron] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:58.763547 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1147/2270] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:31:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:31:59.346069 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06392e0-5c0b-4bc1-a415-80c127007e07 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:31:59.347883 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-b06392e0-5c0b-4bc1-a415-80c127007e07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:31:59.365644 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06392e0-5c0b-4bc1-a415-80c127007e07 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:31:59.377644 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2d1190-3a35-4bd6-834a-cef8206bfbb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:31:59.378248 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2d1190-3a35-4bd6-834a-cef8206bfbb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:31:59.378535 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2d1190-3a35-4bd6-834a-cef8206bfbb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:31:59.379881 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2d1190-3a35-4bd6-834a-cef8206bfbb9 admin admin] PATH_INFO: `/v3/projects/88dcc46eb6634f90b03ec2819becff6a` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:31:59.381755 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2d1190-3a35-4bd6-834a-cef8206bfbb9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=88dcc46eb6634f90b03ec2819becff6a)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:31:59.382632 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2d1190-3a35-4bd6-834a-cef8206bfbb9 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:31:59.553739 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-8e2d1190-3a35-4bd6-834a-cef8206bfbb9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88dcc46eb6634f90b03ec2819becff6a', 'request_id': 'req-8e2d1190-3a35-4bd6-834a-cef8206bfbb9'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:31:59.674868 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-8e2d1190-3a35-4bd6-834a-cef8206bfbb9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88dcc46eb6634f90b03ec2819becff6a', 'request_id': 'req-8e2d1190-3a35-4bd6-834a-cef8206bfbb9'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:31:59.683093 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-8e2d1190-3a35-4bd6-834a-cef8206bfbb9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88dcc46eb6634f90b03ec2819becff6a', 'request_id': 'req-8e2d1190-3a35-4bd6-834a-cef8206bfbb9'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:31:59.695848 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-8e2d1190-3a35-4bd6-834a-cef8206bfbb9 admin admin] The token cache is being invalidate because project 88dcc46eb6634f90b03ec2819becff6a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81732) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:31:59.696099 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-8e2d1190-3a35-4bd6-834a-cef8206bfbb9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:31:59.696996 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-8e2d1190-3a35-4bd6-834a-cef8206bfbb9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:31:59.698613 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1124/2271] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 24 17:31:59 2022] DELETE /identity/v3/projects/88dcc46eb6634f90b03ec2819becff6a => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:32:08.759443 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:32:09.691948 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:32:16.179510 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f53272f-b74a-4f8a-ac2c-8a728ba1e547 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:32:16.180166 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f53272f-b74a-4f8a-ac2c-8a728ba1e547 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:32:16.180166 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f53272f-b74a-4f8a-ac2c-8a728ba1e547 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:32:16.201088 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-2f53272f-b74a-4f8a-ac2c-8a728ba1e547 None None] MFA Rules not processed for user `0ccc6723f7514e52863d102de3839951`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:32:16.202296 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-2f53272f-b74a-4f8a-ac2c-8a728ba1e547 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:32:16.265078 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1148/2272] 10.222.0.18 () {58 vars in 967 bytes} [Wed Aug 24 17:32:16 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:32:16.278166 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2d1190-3a35-4bd6-834a-cef8206bfbb9 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:32:16.279806 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-8e2d1190-3a35-4bd6-834a-cef8206bfbb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:32:16.361501 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2d1190-3a35-4bd6-834a-cef8206bfbb9 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:32:16.368062 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdd3d16-3dc2-41d7-b4c2-f304a9aaf91a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef8caa353be34499a58cd25b72ee4032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:32:16.368695 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdd3d16-3dc2-41d7-b4c2-f304a9aaf91a service swift] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:32:16.368976 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdd3d16-3dc2-41d7-b4c2-f304a9aaf91a service swift] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:32:16.369263 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdd3d16-3dc2-41d7-b4c2-f304a9aaf91a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:32:16.378176 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdd3d16-3dc2-41d7-b4c2-f304a9aaf91a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:32:16.379813 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdd3d16-3dc2-41d7-b4c2-f304a9aaf91a service swift] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:32:16.389130 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1125/2273] 10.222.0.18 () {60 vars in 1384 bytes} [Wed Aug 24 17:32:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 24 17:32:26.256738 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=25 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:32:26.385503 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=32 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:32:26.966114 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f53272f-b74a-4f8a-ac2c-8a728ba1e547 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:32:26.967825 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-2f53272f-b74a-4f8a-ac2c-8a728ba1e547 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:32:27.056035 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f53272f-b74a-4f8a-ac2c-8a728ba1e547 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:32:27.061872 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ca113c-0a55-49f2-8f26-b6e962382c77 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557aca9255cb4bbb8af52e796c5c5177', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:32:27.062478 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ca113c-0a55-49f2-8f26-b6e962382c77 service nova] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:32:27.062777 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ca113c-0a55-49f2-8f26-b6e962382c77 service nova] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:32:27.063074 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ca113c-0a55-49f2-8f26-b6e962382c77 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:32:27.064814 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-82ca113c-0a55-49f2-8f26-b6e962382c77 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:32:27.126478 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ca113c-0a55-49f2-8f26-b6e962382c77 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:32:27.129212 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ca113c-0a55-49f2-8f26-b6e962382c77 service nova] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:32:27.137601 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1149/2274] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:32:26 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:32:37.133374 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=56 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:32:57.640311 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdd3d16-3dc2-41d7-b4c2-f304a9aaf91a service swift] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:32:57.656621 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdd3d16-3dc2-41d7-b4c2-f304a9aaf91a service swift] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:32:57.662562 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4707a1dd-cbcf-46e2-9df9-57b9e5ea155f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557aca9255cb4bbb8af52e796c5c5177', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:32:57.663251 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4707a1dd-cbcf-46e2-9df9-57b9e5ea155f service nova] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:32:57.663568 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4707a1dd-cbcf-46e2-9df9-57b9e5ea155f service nova] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:32:57.663879 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4707a1dd-cbcf-46e2-9df9-57b9e5ea155f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:32:57.666573 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-4707a1dd-cbcf-46e2-9df9-57b9e5ea155f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:32:57.744212 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4707a1dd-cbcf-46e2-9df9-57b9e5ea155f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:32:57.745835 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4707a1dd-cbcf-46e2-9df9-57b9e5ea155f service nova] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:32:57.756184 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1126/2275] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:32:57 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:32:59.436732 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e63495-2759-46a0-b405-27793020e529 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:32:59.437371 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e63495-2759-46a0-b405-27793020e529 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:32:59.437719 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e63495-2759-46a0-b405-27793020e529 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:32:59.464372 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-98e63495-2759-46a0-b405-27793020e529 None None] MFA Rules not processed for user `0696e6f707274727b57a9a7493c47664`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:32:59.466228 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-98e63495-2759-46a0-b405-27793020e529 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:32:59.477081 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1150/2276] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:32:59 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:33:07.751774 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=28 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:33:09.088463 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b4c5d3-e7ea-4cbb-99c3-afaa7606d2d0 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:09.089036 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b4c5d3-e7ea-4cbb-99c3-afaa7606d2d0 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:09.089036 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b4c5d3-e7ea-4cbb-99c3-afaa7606d2d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:09.110276 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-d4b4c5d3-e7ea-4cbb-99c3-afaa7606d2d0 None None] MFA Rules not processed for user `47da700d7d0f4d5cb1ea984a82d643bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:33:09.111500 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-d4b4c5d3-e7ea-4cbb-99c3-afaa7606d2d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:33:09.118454 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1127/2277] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:33:09 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:33:09.456131 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=3 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:33:19.102699 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=3 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:33:38.297344 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e63495-2759-46a0-b405-27793020e529 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:33:38.318756 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e63495-2759-46a0-b405-27793020e529 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:33:38.325828 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4682eda4-66d2-4ba4-bc55-db49e6ae8003 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557aca9255cb4bbb8af52e796c5c5177', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:33:38.326940 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4682eda4-66d2-4ba4-bc55-db49e6ae8003 service nova] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:38.327382 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4682eda4-66d2-4ba4-bc55-db49e6ae8003 service nova] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:38.327827 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4682eda4-66d2-4ba4-bc55-db49e6ae8003 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:38.330332 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-4682eda4-66d2-4ba4-bc55-db49e6ae8003 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:33:38.350748 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4682eda4-66d2-4ba4-bc55-db49e6ae8003 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:33:38.352691 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4682eda4-66d2-4ba4-bc55-db49e6ae8003 service nova] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:33:38.370191 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1151/2278] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:33:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:33:48.362401 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=6 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:33:51.949660 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994ca921-3802-4ccc-8b88-47a34e751297 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:51.949660 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994ca921-3802-4ccc-8b88-47a34e751297 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:51.949660 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994ca921-3802-4ccc-8b88-47a34e751297 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:51.984081 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-994ca921-3802-4ccc-8b88-47a34e751297 None None] MFA Rules not processed for user `c379c845fccd43c08e8699976d506801`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:33:51.985315 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-994ca921-3802-4ccc-8b88-47a34e751297 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:33:52.062706 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1128/2279] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:33:51 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:33:52.268805 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee4b3ab-7ef7-4e98-bb9e-056c94dd1a04 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:33:52.269384 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee4b3ab-7ef7-4e98-bb9e-056c94dd1a04 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:33:52.269786 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee4b3ab-7ef7-4e98-bb9e-056c94dd1a04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:33:52.299722 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-dee4b3ab-7ef7-4e98-bb9e-056c94dd1a04 None None] MFA Rules not processed for user `160ce86c89ea449da346d98d20ce2a9e`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:33:52.300990 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-dee4b3ab-7ef7-4e98-bb9e-056c94dd1a04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:33:52.371971 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1152/2280] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:33:52 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:34:02.059716 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=37 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:02.363266 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=25 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:22.923855 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994ca921-3802-4ccc-8b88-47a34e751297 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:22.925402 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-994ca921-3802-4ccc-8b88-47a34e751297 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:34:23.086032 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994ca921-3802-4ccc-8b88-47a34e751297 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:23.091832 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e650a80a-d0bd-4f72-84e6-f9fb442e1874 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03fc77c6107e430f90f7b6f7cfd4ec02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:23.092384 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e650a80a-d0bd-4f72-84e6-f9fb442e1874 service placement] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:23.092660 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e650a80a-d0bd-4f72-84e6-f9fb442e1874 service placement] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:23.092937 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e650a80a-d0bd-4f72-84e6-f9fb442e1874 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:23.094713 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-e650a80a-d0bd-4f72-84e6-f9fb442e1874 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:34:23.109614 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e650a80a-d0bd-4f72-84e6-f9fb442e1874 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:23.111237 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e650a80a-d0bd-4f72-84e6-f9fb442e1874 service placement] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:23.124930 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1129/2281] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:34:22 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:34:28.231666 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4274c83a-26b4-4bc9-b005-82d78f179bb8 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:28.232296 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4274c83a-26b4-4bc9-b005-82d78f179bb8 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:28.232499 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4274c83a-26b4-4bc9-b005-82d78f179bb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:28.263095 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-4274c83a-26b4-4bc9-b005-82d78f179bb8 None None] MFA Rules not processed for user `ef9e96b709b345c08a4ea10f14211668`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:34:28.264915 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-4274c83a-26b4-4bc9-b005-82d78f179bb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:34:28.331731 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1153/2282] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:34:28 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:34:33.125174 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=51 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:38.325031 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=25 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:52.584188 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e650a80a-d0bd-4f72-84e6-f9fb442e1874 service placement] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:52.585676 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-e650a80a-d0bd-4f72-84e6-f9fb442e1874 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:34:52.710628 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e650a80a-d0bd-4f72-84e6-f9fb442e1874 service placement] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:52.714857 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a9cf62-9da8-4042-8643-45b2912f120b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e9ed4d47aef4140bfb2d3bb798cc559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:52.715502 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a9cf62-9da8-4042-8643-45b2912f120b service neutron] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:52.715782 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a9cf62-9da8-4042-8643-45b2912f120b service neutron] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:52.716056 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a9cf62-9da8-4042-8643-45b2912f120b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:52.717867 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-84a9cf62-9da8-4042-8643-45b2912f120b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:34:52.810302 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a9cf62-9da8-4042-8643-45b2912f120b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:52.811819 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a9cf62-9da8-4042-8643-45b2912f120b service neutron] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:52.832853 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1130/2283] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:34:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:34:57.224424 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4274c83a-26b4-4bc9-b005-82d78f179bb8 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:57.225939 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-4274c83a-26b4-4bc9-b005-82d78f179bb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:34:57.243904 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4274c83a-26b4-4bc9-b005-82d78f179bb8 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:57.249786 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b43e1e-6dc8-4227-b558-3c8bbf5da1cb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03fc77c6107e430f90f7b6f7cfd4ec02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:57.250507 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b43e1e-6dc8-4227-b558-3c8bbf5da1cb service placement] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:57.250950 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b43e1e-6dc8-4227-b558-3c8bbf5da1cb service placement] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:57.251656 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b43e1e-6dc8-4227-b558-3c8bbf5da1cb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:57.254164 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-b5b43e1e-6dc8-4227-b558-3c8bbf5da1cb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:34:57.268023 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b43e1e-6dc8-4227-b558-3c8bbf5da1cb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:57.269614 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b43e1e-6dc8-4227-b558-3c8bbf5da1cb service placement] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:57.281706 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1154/2284] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:34:57 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:34:57.392056 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a9cf62-9da8-4042-8643-45b2912f120b service neutron] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:57.394184 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-84a9cf62-9da8-4042-8643-45b2912f120b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:34:57.411534 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a9cf62-9da8-4042-8643-45b2912f120b service neutron] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:57.420479 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985944d4-82b8-4251-9229-5292074f3a37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:57.421135 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985944d4-82b8-4251-9229-5292074f3a37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:57.421416 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985944d4-82b8-4251-9229-5292074f3a37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:57.421691 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985944d4-82b8-4251-9229-5292074f3a37 admin admin] PATH_INFO: `/v3/users/0696e6f707274727b57a9a7493c47664` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:57.422944 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985944d4-82b8-4251-9229-5292074f3a37 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0696e6f707274727b57a9a7493c47664)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:57.424894 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985944d4-82b8-4251-9229-5292074f3a37 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:57.467668 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aae07fa-1fec-4d2d-b7df-f1849abcd613 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:57.468180 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aae07fa-1fec-4d2d-b7df-f1849abcd613 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:57.468737 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aae07fa-1fec-4d2d-b7df-f1849abcd613 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:57.501971 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-2aae07fa-1fec-4d2d-b7df-f1849abcd613 None None] MFA Rules not processed for user `0ccc6723f7514e52863d102de3839951`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:34:57.503210 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-2aae07fa-1fec-4d2d-b7df-f1849abcd613 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:34:57.517958 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1155/2285] 10.222.0.18 () {58 vars in 967 bytes} [Wed Aug 24 17:34:57 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:34:57.628619 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-985944d4-82b8-4251-9229-5292074f3a37 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0696e6f707274727b57a9a7493c47664'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:34:57.767190 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aae07fa-1fec-4d2d-b7df-f1849abcd613 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:57.772267 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aae07fa-1fec-4d2d-b7df-f1849abcd613 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:57.778063 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2138cc09-f9e1-4246-89c0-34051c9e83b8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef8caa353be34499a58cd25b72ee4032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:57.778671 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2138cc09-f9e1-4246-89c0-34051c9e83b8 service swift] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:57.778976 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2138cc09-f9e1-4246-89c0-34051c9e83b8 service swift] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:57.779251 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2138cc09-f9e1-4246-89c0-34051c9e83b8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:57.787761 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2138cc09-f9e1-4246-89c0-34051c9e83b8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:57.788640 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-985944d4-82b8-4251-9229-5292074f3a37 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0696e6f707274727b57a9a7493c47664'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:34:57.789421 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-985944d4-82b8-4251-9229-5292074f3a37 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0696e6f707274727b57a9a7493c47664'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:34:57.789546 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2138cc09-f9e1-4246-89c0-34051c9e83b8 service swift] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:57.793238 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:57.798463 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1156/2286] 10.222.0.18 () {60 vars in 1384 bytes} [Wed Aug 24 17:34:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 24 17:34:57.801089 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-985944d4-82b8-4251-9229-5292074f3a37 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0696e6f707274727b57a9a7493c47664'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:34:57.802734 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-985944d4-82b8-4251-9229-5292074f3a37 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0696e6f707274727b57a9a7493c47664'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:34:57.824792 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1131/2287] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 24 17:34:57 2022] DELETE /identity/v3/users/0696e6f707274727b57a9a7493c47664 => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:34:58.113435 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2138cc09-f9e1-4246-89c0-34051c9e83b8 service swift] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:58.127590 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2138cc09-f9e1-4246-89c0-34051c9e83b8 service swift] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:58.209972 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206195a8-1083-4fc4-b7c4-7c0674911cb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:58.210579 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206195a8-1083-4fc4-b7c4-7c0674911cb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:58.210920 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206195a8-1083-4fc4-b7c4-7c0674911cb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:58.211314 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206195a8-1083-4fc4-b7c4-7c0674911cb8 admin admin] PATH_INFO: `/v3/projects/cf9b8b1e5c0f4021ae5f40c11c962222` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:58.212503 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206195a8-1083-4fc4-b7c4-7c0674911cb8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf9b8b1e5c0f4021ae5f40c11c962222)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:58.214416 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206195a8-1083-4fc4-b7c4-7c0674911cb8 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:58.390259 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-206195a8-1083-4fc4-b7c4-7c0674911cb8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf9b8b1e5c0f4021ae5f40c11c962222', 'request_id': 'req-206195a8-1083-4fc4-b7c4-7c0674911cb8'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:34:58.487357 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-206195a8-1083-4fc4-b7c4-7c0674911cb8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf9b8b1e5c0f4021ae5f40c11c962222', 'request_id': 'req-206195a8-1083-4fc4-b7c4-7c0674911cb8'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:34:58.499291 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-206195a8-1083-4fc4-b7c4-7c0674911cb8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf9b8b1e5c0f4021ae5f40c11c962222', 'request_id': 'req-206195a8-1083-4fc4-b7c4-7c0674911cb8'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:34:58.521341 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-206195a8-1083-4fc4-b7c4-7c0674911cb8 admin admin] The token cache is being invalidate because project cf9b8b1e5c0f4021ae5f40c11c962222 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81731) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:34:58.521711 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-206195a8-1083-4fc4-b7c4-7c0674911cb8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:34:58.522457 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-206195a8-1083-4fc4-b7c4-7c0674911cb8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:34:58.524920 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1157/2288] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 24 17:34:58 2022] DELETE /identity/v3/projects/cf9b8b1e5c0f4021ae5f40c11c962222 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:34:58.531856 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7bb2b1-f903-4943-8629-f2478ae117ed None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:58.533241 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7bb2b1-f903-4943-8629-f2478ae117ed None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:58.533433 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7bb2b1-f903-4943-8629-f2478ae117ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:58.607633 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-3e7bb2b1-f903-4943-8629-f2478ae117ed None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:34:58.608885 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-3e7bb2b1-f903-4943-8629-f2478ae117ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:34:58.685265 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1132/2289] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:34:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:34:58.694409 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a70a506-6a6e-4d25-a53c-5bcb383d1765 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:58.694696 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a70a506-6a6e-4d25-a53c-5bcb383d1765 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:58.695007 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a70a506-6a6e-4d25-a53c-5bcb383d1765 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:58.719602 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-7a70a506-6a6e-4d25-a53c-5bcb383d1765 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:34:58.720831 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-7a70a506-6a6e-4d25-a53c-5bcb383d1765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:34:58.729631 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1158/2290] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:34:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:34:58.734789 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7bb2b1-f903-4943-8629-f2478ae117ed None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:58.748019 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7bb2b1-f903-4943-8629-f2478ae117ed None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:58.751635 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81d84b1-061e-460e-93d2-6eab45625a5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:58.752424 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81d84b1-061e-460e-93d2-6eab45625a5c admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:58.752708 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81d84b1-061e-460e-93d2-6eab45625a5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:58.752982 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81d84b1-061e-460e-93d2-6eab45625a5c admin admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:58.753644 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81d84b1-061e-460e-93d2-6eab45625a5c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:34:58.753948 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81d84b1-061e-460e-93d2-6eab45625a5c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:58.755666 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81d84b1-061e-460e-93d2-6eab45625a5c admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:58.776186 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1133/2291] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 24 17:34:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:34:58.784176 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a70a506-6a6e-4d25-a53c-5bcb383d1765 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:58.803081 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a70a506-6a6e-4d25-a53c-5bcb383d1765 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:58.810395 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22ba899-abd6-42d0-acaa-0749f5215229 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:58.811296 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22ba899-abd6-42d0-acaa-0749f5215229 admin admin] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:58.816929 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22ba899-abd6-42d0-acaa-0749f5215229 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:58.816929 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22ba899-abd6-42d0-acaa-0749f5215229 admin admin] PATH_INFO: `/v3/projects` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:58.816929 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22ba899-abd6-42d0-acaa-0749f5215229 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:58.816929 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22ba899-abd6-42d0-acaa-0749f5215229 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:58.913671 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1159/2292] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 24 17:34:58 2022] POST /identity/v3/projects => generated 371 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:34:58.919591 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81d84b1-061e-460e-93d2-6eab45625a5c admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:58.930267 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81d84b1-061e-460e-93d2-6eab45625a5c admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:59.016113 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7074df95-186a-4344-a03f-75d24773b7c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:59.016860 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7074df95-186a-4344-a03f-75d24773b7c8 admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:59.017275 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7074df95-186a-4344-a03f-75d24773b7c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:59.017981 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7074df95-186a-4344-a03f-75d24773b7c8 admin admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:59.018892 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7074df95-186a-4344-a03f-75d24773b7c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:59.025314 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7074df95-186a-4344-a03f-75d24773b7c8 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:59.110356 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1134/2293] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 24 17:34:58 2022] POST /identity/v3/users => generated 286 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:34:59.120808 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22ba899-abd6-42d0-acaa-0749f5215229 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:59.132264 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22ba899-abd6-42d0-acaa-0749f5215229 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:59.139621 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a0cdf1-ce93-4f99-ae7b-8d339b0e50cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:59.140337 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a0cdf1-ce93-4f99-ae7b-8d339b0e50cf admin admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:59.140658 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a0cdf1-ce93-4f99-ae7b-8d339b0e50cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:59.141073 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a0cdf1-ce93-4f99-ae7b-8d339b0e50cf admin admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:59.141821 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a0cdf1-ce93-4f99-ae7b-8d339b0e50cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:59.143676 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a0cdf1-ce93-4f99-ae7b-8d339b0e50cf admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:59.154807 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1160/2294] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:34:59 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:34:59.162282 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7074df95-186a-4344-a03f-75d24773b7c8 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:59.177504 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7074df95-186a-4344-a03f-75d24773b7c8 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:59.182971 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5525909-83a9-45bb-b7a3-cdfdef4a471b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:59.183631 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5525909-83a9-45bb-b7a3-cdfdef4a471b admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:59.183899 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5525909-83a9-45bb-b7a3-cdfdef4a471b admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:59.184168 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5525909-83a9-45bb-b7a3-cdfdef4a471b admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:59.184821 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5525909-83a9-45bb-b7a3-cdfdef4a471b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:59.186519 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5525909-83a9-45bb-b7a3-cdfdef4a471b admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:59.198388 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1135/2295] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:34:59 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:34:59.209829 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a0cdf1-ce93-4f99-ae7b-8d339b0e50cf admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:59.209829 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:59.224392 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a0cdf1-ce93-4f99-ae7b-8d339b0e50cf admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:59.232463 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5525909-83a9-45bb-b7a3-cdfdef4a471b admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:59.234497 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-e5525909-83a9-45bb-b7a3-cdfdef4a471b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:34:59.235982 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172bc1c7-3761-4eb2-a080-b4a9ddf1aecd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:59.236924 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172bc1c7-3761-4eb2-a080-b4a9ddf1aecd admin admin] REQUEST_METHOD: `PUT` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:59.237338 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172bc1c7-3761-4eb2-a080-b4a9ddf1aecd admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:59.237499 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172bc1c7-3761-4eb2-a080-b4a9ddf1aecd admin admin] PATH_INFO: `/v3/projects/39936f4fa2c04ddba1fa0f318dd69490/users/23413655643b47709304a632ec01d083/roles/60dc7e16aee746d5a3e8aa27d7d8525c` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:59.253160 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:34:59.256374 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172bc1c7-3761-4eb2-a080-b4a9ddf1aecd admin admin] RBAC: Authorizing `identity:create_grant(project_id=39936f4fa2c04ddba1fa0f318dd69490, user_id=23413655643b47709304a632ec01d083, role_id=60dc7e16aee746d5a3e8aa27d7d8525c)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:59.260045 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172bc1c7-3761-4eb2-a080-b4a9ddf1aecd admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:59.315179 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5525909-83a9-45bb-b7a3-cdfdef4a471b admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:59.321532 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c85b0c2-ce43-4ccc-915e-597e3ad2d08e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e9ed4d47aef4140bfb2d3bb798cc559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:59.322135 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c85b0c2-ce43-4ccc-915e-597e3ad2d08e service neutron] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:59.322515 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c85b0c2-ce43-4ccc-915e-597e3ad2d08e service neutron] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:59.322958 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c85b0c2-ce43-4ccc-915e-597e3ad2d08e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:59.325022 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-9c85b0c2-ce43-4ccc-915e-597e3ad2d08e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:34:59.343674 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c85b0c2-ce43-4ccc-915e-597e3ad2d08e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:59.345562 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c85b0c2-ce43-4ccc-915e-597e3ad2d08e service neutron] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:59.348693 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1161/2296] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 24 17:34:59 2022] PUT /identity/v3/projects/39936f4fa2c04ddba1fa0f318dd69490/users/23413655643b47709304a632ec01d083/roles/60dc7e16aee746d5a3e8aa27d7d8525c => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:34:59.357103 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1136/2297] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:34:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:34:59.360344 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429665df-dd1b-457d-bd38-6609c90e81db None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:59.360790 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429665df-dd1b-457d-bd38-6609c90e81db None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:59.361074 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429665df-dd1b-457d-bd38-6609c90e81db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:59.397773 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-429665df-dd1b-457d-bd38-6609c90e81db None None] MFA Rules not processed for user `23413655643b47709304a632ec01d083`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:34:59.399643 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-429665df-dd1b-457d-bd38-6609c90e81db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:34:59.491610 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1162/2298] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:34:59 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:34:59.498008 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c85b0c2-ce43-4ccc-915e-597e3ad2d08e service neutron] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:59.511611 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c85b0c2-ce43-4ccc-915e-597e3ad2d08e service neutron] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:59.613119 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a77c31-3321-4e42-9553-748ccf519e52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:59.614200 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a77c31-3321-4e42-9553-748ccf519e52 admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:59.615131 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a77c31-3321-4e42-9553-748ccf519e52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:59.615131 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a77c31-3321-4e42-9553-748ccf519e52 admin admin] PATH_INFO: `/v3/projects` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:59.616022 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a77c31-3321-4e42-9553-748ccf519e52 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:59.618491 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a77c31-3321-4e42-9553-748ccf519e52 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:59.733591 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1137/2299] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 24 17:34:59 2022] POST /identity/v3/projects => generated 371 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:34:59.738025 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429665df-dd1b-457d-bd38-6609c90e81db None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:59.759322 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429665df-dd1b-457d-bd38-6609c90e81db None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:34:59.849751 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdad24c8-851c-4329-ba13-9e95df17ef4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:34:59.850810 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdad24c8-851c-4329-ba13-9e95df17ef4c admin admin] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:34:59.851342 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdad24c8-851c-4329-ba13-9e95df17ef4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:34:59.851712 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdad24c8-851c-4329-ba13-9e95df17ef4c admin admin] PATH_INFO: `/v3/users` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:34:59.852894 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdad24c8-851c-4329-ba13-9e95df17ef4c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:34:59.856639 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdad24c8-851c-4329-ba13-9e95df17ef4c admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:34:59.983923 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1163/2300] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 24 17:34:59 2022] POST /identity/v3/users => generated 292 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:34:59.986545 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a77c31-3321-4e42-9553-748ccf519e52 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:34:59.997396 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a77c31-3321-4e42-9553-748ccf519e52 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:00.002964 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49839b9e-b1cc-4838-9950-6628e61c197d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:00.003915 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49839b9e-b1cc-4838-9950-6628e61c197d admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:00.004315 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49839b9e-b1cc-4838-9950-6628e61c197d admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:00.009658 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49839b9e-b1cc-4838-9950-6628e61c197d admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:00.009658 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49839b9e-b1cc-4838-9950-6628e61c197d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:00.009658 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49839b9e-b1cc-4838-9950-6628e61c197d admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:00.036099 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1138/2301] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:34:59 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:35:00.038379 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdad24c8-851c-4329-ba13-9e95df17ef4c admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:00.049301 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdad24c8-851c-4329-ba13-9e95df17ef4c admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:00.054675 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5e4348-84de-4485-89a4-a354fc8efe2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:00.056224 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5e4348-84de-4485-89a4-a354fc8efe2d admin admin] REQUEST_METHOD: `PUT` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:00.095350 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5e4348-84de-4485-89a4-a354fc8efe2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:00.095350 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5e4348-84de-4485-89a4-a354fc8efe2d admin admin] PATH_INFO: `/v3/projects/10dbc3f9674b4190be573f991de51cec/users/f4f6c5d0bd2946a2b7b5c5c4988b9bf3/roles/c5346bc54abf445f9900edf94264c850` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:00.095350 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5e4348-84de-4485-89a4-a354fc8efe2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=10dbc3f9674b4190be573f991de51cec, user_id=f4f6c5d0bd2946a2b7b5c5c4988b9bf3, role_id=c5346bc54abf445f9900edf94264c850)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:00.095350 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5e4348-84de-4485-89a4-a354fc8efe2d admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:00.189555 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1164/2302] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 24 17:35:00 2022] PUT /identity/v3/projects/10dbc3f9674b4190be573f991de51cec/users/f4f6c5d0bd2946a2b7b5c5c4988b9bf3/roles/c5346bc54abf445f9900edf94264c850 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:35:00.194430 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49839b9e-b1cc-4838-9950-6628e61c197d admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:00.207772 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49839b9e-b1cc-4838-9950-6628e61c197d admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:00.280313 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dee172-0b57-4d3d-8895-b5bfa71c7e8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:00.280943 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dee172-0b57-4d3d-8895-b5bfa71c7e8e admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:00.281225 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dee172-0b57-4d3d-8895-b5bfa71c7e8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:00.281507 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dee172-0b57-4d3d-8895-b5bfa71c7e8e admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:00.282183 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91dee172-0b57-4d3d-8895-b5bfa71c7e8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:00.283850 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91dee172-0b57-4d3d-8895-b5bfa71c7e8e admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:00.296152 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1139/2303] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:35:00 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:35:00.300826 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5e4348-84de-4485-89a4-a354fc8efe2d admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:00.312018 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5e4348-84de-4485-89a4-a354fc8efe2d admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:00.322756 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f130539-c184-4342-a8a1-545e7ff5c486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:00.322756 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f130539-c184-4342-a8a1-545e7ff5c486 admin admin] REQUEST_METHOD: `PUT` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:00.322756 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f130539-c184-4342-a8a1-545e7ff5c486 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:00.322756 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f130539-c184-4342-a8a1-545e7ff5c486 admin admin] PATH_INFO: `/v3/projects/10dbc3f9674b4190be573f991de51cec/users/f4f6c5d0bd2946a2b7b5c5c4988b9bf3/roles/c5346bc54abf445f9900edf94264c850` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:00.323255 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f130539-c184-4342-a8a1-545e7ff5c486 admin admin] RBAC: Authorizing `identity:create_grant(project_id=10dbc3f9674b4190be573f991de51cec, user_id=f4f6c5d0bd2946a2b7b5c5c4988b9bf3, role_id=c5346bc54abf445f9900edf94264c850)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:00.323772 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f130539-c184-4342-a8a1-545e7ff5c486 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:00.337892 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1165/2304] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 24 17:35:00 2022] PUT /identity/v3/projects/10dbc3f9674b4190be573f991de51cec/users/f4f6c5d0bd2946a2b7b5c5c4988b9bf3/roles/c5346bc54abf445f9900edf94264c850 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:35:00.349586 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10e8c55-aaaf-4c9e-89cd-a7783e7a8667 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:00.351316 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10e8c55-aaaf-4c9e-89cd-a7783e7a8667 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:00.351316 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10e8c55-aaaf-4c9e-89cd-a7783e7a8667 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:00.380569 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-c10e8c55-aaaf-4c9e-89cd-a7783e7a8667 None None] MFA Rules not processed for user `f4f6c5d0bd2946a2b7b5c5c4988b9bf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:35:00.381620 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-c10e8c55-aaaf-4c9e-89cd-a7783e7a8667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:35:00.391901 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:00.467690 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1140/2305] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:35:00 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:35:00.482646 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f130539-c184-4342-a8a1-545e7ff5c486 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:00.484815 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-5f130539-c184-4342-a8a1-545e7ff5c486 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:35:00.514170 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:00.603148 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f130539-c184-4342-a8a1-545e7ff5c486 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:00.611056 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3877b939-f853-43d4-bb55-9d0a70074ae7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be06246d47584b529150ceb98a5cfd95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:00.612130 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3877b939-f853-43d4-bb55-9d0a70074ae7 service cinder] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:00.612616 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3877b939-f853-43d4-bb55-9d0a70074ae7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:00.613055 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3877b939-f853-43d4-bb55-9d0a70074ae7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:00.628753 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3877b939-f853-43d4-bb55-9d0a70074ae7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:00.630935 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3877b939-f853-43d4-bb55-9d0a70074ae7 service cinder] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:00.644955 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1166/2306] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:35:00 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:35:00.927809 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10e8c55-aaaf-4c9e-89cd-a7783e7a8667 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:00.938987 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10e8c55-aaaf-4c9e-89cd-a7783e7a8667 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:00.943232 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6415082-6828-4130-bda9-5c26f49d5710 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be06246d47584b529150ceb98a5cfd95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:00.943895 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6415082-6828-4130-bda9-5c26f49d5710 service cinder] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:00.944183 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6415082-6828-4130-bda9-5c26f49d5710 service cinder] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:00.944559 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6415082-6828-4130-bda9-5c26f49d5710 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:01.052140 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6415082-6828-4130-bda9-5c26f49d5710 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:01.053704 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6415082-6828-4130-bda9-5c26f49d5710 service cinder] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:01.067066 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1141/2307] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:35:00 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:35:03.876963 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3877b939-f853-43d4-bb55-9d0a70074ae7 service cinder] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:03.878715 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-3877b939-f853-43d4-bb55-9d0a70074ae7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:35:03.969926 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3877b939-f853-43d4-bb55-9d0a70074ae7 service cinder] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:03.975645 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed0c19c-4a7c-488c-9ce0-4be3905c0e2d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03fc77c6107e430f90f7b6f7cfd4ec02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:03.976360 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed0c19c-4a7c-488c-9ce0-4be3905c0e2d service placement] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:03.977362 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed0c19c-4a7c-488c-9ce0-4be3905c0e2d service placement] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:03.977665 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed0c19c-4a7c-488c-9ce0-4be3905c0e2d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:03.979733 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-3ed0c19c-4a7c-488c-9ce0-4be3905c0e2d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:35:03.992117 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed0c19c-4a7c-488c-9ce0-4be3905c0e2d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:03.993911 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed0c19c-4a7c-488c-9ce0-4be3905c0e2d service placement] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:04.004968 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1167/2308] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:35:03 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:35:08.290501 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6415082-6828-4130-bda9-5c26f49d5710 service cinder] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:08.291593 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-d6415082-6828-4130-bda9-5c26f49d5710 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:35:08.365191 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6415082-6828-4130-bda9-5c26f49d5710 service cinder] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:08.370815 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ea9b5f-6189-40ea-ad91-c9fe2aca4849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:08.371577 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ea9b5f-6189-40ea-ad91-c9fe2aca4849 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:08.371857 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ea9b5f-6189-40ea-ad91-c9fe2aca4849 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:08.372131 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ea9b5f-6189-40ea-ad91-c9fe2aca4849 admin admin] PATH_INFO: `/v3/users/78ce397b5ece4c5b81a1bbefcf87df9d` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:08.373313 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ea9b5f-6189-40ea-ad91-c9fe2aca4849 admin admin] RBAC: Authorizing `identity:delete_user(user_id=78ce397b5ece4c5b81a1bbefcf87df9d)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:08.375400 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ea9b5f-6189-40ea-ad91-c9fe2aca4849 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:08.557628 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-81ea9b5f-6189-40ea-ad91-c9fe2aca4849 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '78ce397b5ece4c5b81a1bbefcf87df9d'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:35:08.559541 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:08.631200 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-81ea9b5f-6189-40ea-ad91-c9fe2aca4849 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '78ce397b5ece4c5b81a1bbefcf87df9d'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:35:08.631983 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-81ea9b5f-6189-40ea-ad91-c9fe2aca4849 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '78ce397b5ece4c5b81a1bbefcf87df9d'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:35:08.641681 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-81ea9b5f-6189-40ea-ad91-c9fe2aca4849 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '78ce397b5ece4c5b81a1bbefcf87df9d'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:35:08.642587 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-81ea9b5f-6189-40ea-ad91-c9fe2aca4849 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '78ce397b5ece4c5b81a1bbefcf87df9d'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:35:08.662424 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1142/2309] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 24 17:35:08 2022] DELETE /identity/v3/users/78ce397b5ece4c5b81a1bbefcf87df9d => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:35:08.923954 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed0c19c-4a7c-488c-9ce0-4be3905c0e2d service placement] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:08.936297 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed0c19c-4a7c-488c-9ce0-4be3905c0e2d service placement] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:09.002802 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaa5917-647b-4850-8616-4c2f86cdff58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:09.003570 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaa5917-647b-4850-8616-4c2f86cdff58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:09.003847 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaa5917-647b-4850-8616-4c2f86cdff58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:09.004121 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaa5917-647b-4850-8616-4c2f86cdff58 admin admin] PATH_INFO: `/v3/projects/95cbbaf120e848a0a699fd6391aebcfd` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:09.005362 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abaa5917-647b-4850-8616-4c2f86cdff58 admin admin] RBAC: Authorizing `identity:delete_project(project_id=95cbbaf120e848a0a699fd6391aebcfd)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:09.007406 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abaa5917-647b-4850-8616-4c2f86cdff58 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:09.103573 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:09.116159 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-abaa5917-647b-4850-8616-4c2f86cdff58 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95cbbaf120e848a0a699fd6391aebcfd', 'request_id': 'req-abaa5917-647b-4850-8616-4c2f86cdff58'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:35:09.208426 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-abaa5917-647b-4850-8616-4c2f86cdff58 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95cbbaf120e848a0a699fd6391aebcfd', 'request_id': 'req-abaa5917-647b-4850-8616-4c2f86cdff58'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:35:09.216362 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-abaa5917-647b-4850-8616-4c2f86cdff58 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95cbbaf120e848a0a699fd6391aebcfd', 'request_id': 'req-abaa5917-647b-4850-8616-4c2f86cdff58'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:35:09.227956 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-abaa5917-647b-4850-8616-4c2f86cdff58 admin admin] The token cache is being invalidate because project 95cbbaf120e848a0a699fd6391aebcfd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81731) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:35:09.228253 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-abaa5917-647b-4850-8616-4c2f86cdff58 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:35:09.228943 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-abaa5917-647b-4850-8616-4c2f86cdff58 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:35:09.230440 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1168/2310] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 24 17:35:08 2022] DELETE /identity/v3/projects/95cbbaf120e848a0a699fd6391aebcfd => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:35:09.234681 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ea9b5f-6189-40ea-ad91-c9fe2aca4849 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:09.240360 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-81ea9b5f-6189-40ea-ad91-c9fe2aca4849 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:35:09.250532 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ea9b5f-6189-40ea-ad91-c9fe2aca4849 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:09.321721 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb9f66c-32f3-4d1e-8cb7-9706e088ee32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:09.322846 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb9f66c-32f3-4d1e-8cb7-9706e088ee32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:09.323389 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb9f66c-32f3-4d1e-8cb7-9706e088ee32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:09.323748 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb9f66c-32f3-4d1e-8cb7-9706e088ee32 admin admin] PATH_INFO: `/v3/users/1bc9ddfcbd3c44aa8a99823fd1b6bf97` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:09.324956 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb9f66c-32f3-4d1e-8cb7-9706e088ee32 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1bc9ddfcbd3c44aa8a99823fd1b6bf97)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:09.326888 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb9f66c-32f3-4d1e-8cb7-9706e088ee32 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:09.568790 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-2bb9f66c-32f3-4d1e-8cb7-9706e088ee32 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1bc9ddfcbd3c44aa8a99823fd1b6bf97'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:35:09.659598 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-2bb9f66c-32f3-4d1e-8cb7-9706e088ee32 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1bc9ddfcbd3c44aa8a99823fd1b6bf97'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:35:09.660294 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-2bb9f66c-32f3-4d1e-8cb7-9706e088ee32 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1bc9ddfcbd3c44aa8a99823fd1b6bf97'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:35:09.668622 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-2bb9f66c-32f3-4d1e-8cb7-9706e088ee32 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1bc9ddfcbd3c44aa8a99823fd1b6bf97'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:35:09.669345 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-2bb9f66c-32f3-4d1e-8cb7-9706e088ee32 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1bc9ddfcbd3c44aa8a99823fd1b6bf97'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:35:09.689243 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1143/2311] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 24 17:35:09 2022] DELETE /identity/v3/users/1bc9ddfcbd3c44aa8a99823fd1b6bf97 => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:35:09.964286 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaa5917-647b-4850-8616-4c2f86cdff58 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:09.965782 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-abaa5917-647b-4850-8616-4c2f86cdff58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:35:09.979068 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaa5917-647b-4850-8616-4c2f86cdff58 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:10.038731 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386df54b-f325-41c1-ab2d-e7852725516d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:10.039424 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386df54b-f325-41c1-ab2d-e7852725516d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:10.039704 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386df54b-f325-41c1-ab2d-e7852725516d admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:10.039976 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386df54b-f325-41c1-ab2d-e7852725516d admin admin] PATH_INFO: `/v3/projects/cd2951b036184d1c9d4a75f4bd31aa7d` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:10.041235 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386df54b-f325-41c1-ab2d-e7852725516d admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd2951b036184d1c9d4a75f4bd31aa7d)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:10.043188 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386df54b-f325-41c1-ab2d-e7852725516d admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:10.193213 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-386df54b-f325-41c1-ab2d-e7852725516d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd2951b036184d1c9d4a75f4bd31aa7d', 'request_id': 'req-386df54b-f325-41c1-ab2d-e7852725516d'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:35:10.262246 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-386df54b-f325-41c1-ab2d-e7852725516d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd2951b036184d1c9d4a75f4bd31aa7d', 'request_id': 'req-386df54b-f325-41c1-ab2d-e7852725516d'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:35:10.269430 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-386df54b-f325-41c1-ab2d-e7852725516d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd2951b036184d1c9d4a75f4bd31aa7d', 'request_id': 'req-386df54b-f325-41c1-ab2d-e7852725516d'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:35:10.280565 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-386df54b-f325-41c1-ab2d-e7852725516d admin admin] The token cache is being invalidate because project cd2951b036184d1c9d4a75f4bd31aa7d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81731) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:35:10.284290 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-386df54b-f325-41c1-ab2d-e7852725516d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:35:10.284290 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-386df54b-f325-41c1-ab2d-e7852725516d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:35:10.284290 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1169/2312] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 24 17:35:09 2022] DELETE /identity/v3/projects/cd2951b036184d1c9d4a75f4bd31aa7d => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:35:10.288205 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0105d3d6-c016-48f4-8550-2390485d31e6 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:10.288492 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0105d3d6-c016-48f4-8550-2390485d31e6 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:10.288763 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0105d3d6-c016-48f4-8550-2390485d31e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:10.309243 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-0105d3d6-c016-48f4-8550-2390485d31e6 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:35:10.310448 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-0105d3d6-c016-48f4-8550-2390485d31e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:35:10.350859 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:10.371277 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1144/2313] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:35:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:35:10.379113 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec9ff61-ae3f-4edc-9e2a-a75e65d23586 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:10.379504 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec9ff61-ae3f-4edc-9e2a-a75e65d23586 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:10.379782 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec9ff61-ae3f-4edc-9e2a-a75e65d23586 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:10.402357 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-dec9ff61-ae3f-4edc-9e2a-a75e65d23586 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:35:10.403580 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-dec9ff61-ae3f-4edc-9e2a-a75e65d23586 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:35:10.411420 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1170/2314] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:35:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:35:10.416262 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0105d3d6-c016-48f4-8550-2390485d31e6 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:10.425995 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0105d3d6-c016-48f4-8550-2390485d31e6 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:10.430864 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18be4cd-b81b-499a-be77-fc57ae1605b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:10.431484 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18be4cd-b81b-499a-be77-fc57ae1605b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:10.431760 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18be4cd-b81b-499a-be77-fc57ae1605b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:10.432030 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18be4cd-b81b-499a-be77-fc57ae1605b6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:10.432883 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18be4cd-b81b-499a-be77-fc57ae1605b6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:35:10.433078 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18be4cd-b81b-499a-be77-fc57ae1605b6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:10.434666 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18be4cd-b81b-499a-be77-fc57ae1605b6 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:10.447782 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1145/2315] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 24 17:35:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:35:10.452406 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec9ff61-ae3f-4edc-9e2a-a75e65d23586 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:10.462377 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec9ff61-ae3f-4edc-9e2a-a75e65d23586 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:10.467108 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9ed6bd-999e-4ebd-a1ba-1a6f0f4e09f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:10.467713 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9ed6bd-999e-4ebd-a1ba-1a6f0f4e09f7 admin admin] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:10.467991 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9ed6bd-999e-4ebd-a1ba-1a6f0f4e09f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:10.468262 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9ed6bd-999e-4ebd-a1ba-1a6f0f4e09f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:10.468884 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9ed6bd-999e-4ebd-a1ba-1a6f0f4e09f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:10.470772 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9ed6bd-999e-4ebd-a1ba-1a6f0f4e09f7 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:10.596338 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1171/2316] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 24 17:35:10 2022] POST /identity/v3/projects => generated 373 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:35:10.601679 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18be4cd-b81b-499a-be77-fc57ae1605b6 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:10.611379 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18be4cd-b81b-499a-be77-fc57ae1605b6 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:10.673282 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c46ce43-3d7a-4f08-9f70-81d17e9f5fc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:10.673890 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c46ce43-3d7a-4f08-9f70-81d17e9f5fc6 admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:10.674168 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c46ce43-3d7a-4f08-9f70-81d17e9f5fc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:10.674435 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c46ce43-3d7a-4f08-9f70-81d17e9f5fc6 admin admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:10.675081 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c46ce43-3d7a-4f08-9f70-81d17e9f5fc6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:10.677022 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c46ce43-3d7a-4f08-9f70-81d17e9f5fc6 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:10.805696 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1146/2317] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 24 17:35:10 2022] POST /identity/v3/users => generated 287 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:35:10.807047 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9ed6bd-999e-4ebd-a1ba-1a6f0f4e09f7 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:10.819252 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9ed6bd-999e-4ebd-a1ba-1a6f0f4e09f7 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:10.824747 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293693f3-51fc-46d2-9af1-25bf12c9cb0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:10.825458 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293693f3-51fc-46d2-9af1-25bf12c9cb0a admin admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:10.825740 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293693f3-51fc-46d2-9af1-25bf12c9cb0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:10.826012 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293693f3-51fc-46d2-9af1-25bf12c9cb0a admin admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:10.826683 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293693f3-51fc-46d2-9af1-25bf12c9cb0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:10.828310 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293693f3-51fc-46d2-9af1-25bf12c9cb0a admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:10.838945 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1172/2318] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:35:10 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:35:10.843209 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c46ce43-3d7a-4f08-9f70-81d17e9f5fc6 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:10.853650 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c46ce43-3d7a-4f08-9f70-81d17e9f5fc6 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:10.858589 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5b2581-62d7-4f7a-80c3-58dbdf52bf9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:10.859224 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5b2581-62d7-4f7a-80c3-58dbdf52bf9c admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:10.859511 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5b2581-62d7-4f7a-80c3-58dbdf52bf9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:10.859785 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5b2581-62d7-4f7a-80c3-58dbdf52bf9c admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:10.860430 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5b2581-62d7-4f7a-80c3-58dbdf52bf9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:10.862028 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5b2581-62d7-4f7a-80c3-58dbdf52bf9c admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:10.873526 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1147/2319] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:35:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:35:10.877742 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293693f3-51fc-46d2-9af1-25bf12c9cb0a admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:10.888873 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293693f3-51fc-46d2-9af1-25bf12c9cb0a admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:10.893998 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c8ce19-81cc-4591-8893-35b4554ac9a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:10.894663 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c8ce19-81cc-4591-8893-35b4554ac9a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:10.894971 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c8ce19-81cc-4591-8893-35b4554ac9a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:10.895258 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c8ce19-81cc-4591-8893-35b4554ac9a0 admin admin] PATH_INFO: `/v3/projects/2a66fdbac83c47b59c88adbe6b82ff8a/users/4df56db015514caa8aa92ffcfe227914/roles/60dc7e16aee746d5a3e8aa27d7d8525c` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:10.907565 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:DELETE=12,keystone:SELECT=85,keystone:INSERT=3 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:10.920666 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c8ce19-81cc-4591-8893-35b4554ac9a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a66fdbac83c47b59c88adbe6b82ff8a, user_id=4df56db015514caa8aa92ffcfe227914, role_id=60dc7e16aee746d5a3e8aa27d7d8525c)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:10.920666 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c8ce19-81cc-4591-8893-35b4554ac9a0 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:11.015009 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1173/2320] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 24 17:35:10 2022] PUT /identity/v3/projects/2a66fdbac83c47b59c88adbe6b82ff8a/users/4df56db015514caa8aa92ffcfe227914/roles/60dc7e16aee746d5a3e8aa27d7d8525c => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:35:11.020069 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f70e89c-b0d2-49f3-b335-7a71beaec430 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:11.020336 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f70e89c-b0d2-49f3-b335-7a71beaec430 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:11.020613 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f70e89c-b0d2-49f3-b335-7a71beaec430 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:11.039613 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-7f70e89c-b0d2-49f3-b335-7a71beaec430 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:35:11.040870 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-7f70e89c-b0d2-49f3-b335-7a71beaec430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:35:11.105500 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1148/2321] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:35:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:35:11.118641 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c8ce19-81cc-4591-8893-35b4554ac9a0 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:11.121501 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-d8c8ce19-81cc-4591-8893-35b4554ac9a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:35:11.186547 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c8ce19-81cc-4591-8893-35b4554ac9a0 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:11.192284 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeee7f02-e5bb-42f0-81c7-49de42c3572b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e9ed4d47aef4140bfb2d3bb798cc559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:11.192698 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeee7f02-e5bb-42f0-81c7-49de42c3572b service neutron] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:11.192952 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeee7f02-e5bb-42f0-81c7-49de42c3572b service neutron] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:11.193257 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeee7f02-e5bb-42f0-81c7-49de42c3572b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:11.205397 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeee7f02-e5bb-42f0-81c7-49de42c3572b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:11.207043 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeee7f02-e5bb-42f0-81c7-49de42c3572b service neutron] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:11.217031 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1174/2322] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:35:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:35:17.792322 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9b0361-9dab-4b59-b3a6-ef12f7af690e None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:17.792881 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9b0361-9dab-4b59-b3a6-ef12f7af690e None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:17.792881 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9b0361-9dab-4b59-b3a6-ef12f7af690e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:17.829335 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-9e9b0361-9dab-4b59-b3a6-ef12f7af690e None None] MFA Rules not processed for user `4df56db015514caa8aa92ffcfe227914`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:35:17.830577 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-9e9b0361-9dab-4b59-b3a6-ef12f7af690e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:35:17.877263 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:17.903928 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1149/2323] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:35:17 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:35:17.909190 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeee7f02-e5bb-42f0-81c7-49de42c3572b service neutron] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:17.921246 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeee7f02-e5bb-42f0-81c7-49de42c3572b service neutron] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:17.926623 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71a33cb-d052-4ab9-9054-50fdbb2a58aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:17.927281 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71a33cb-d052-4ab9-9054-50fdbb2a58aa admin admin] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:17.927567 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71a33cb-d052-4ab9-9054-50fdbb2a58aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:17.931708 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71a33cb-d052-4ab9-9054-50fdbb2a58aa admin admin] PATH_INFO: `/v3/projects` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:17.931708 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71a33cb-d052-4ab9-9054-50fdbb2a58aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:17.931708 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71a33cb-d052-4ab9-9054-50fdbb2a58aa admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:18.082721 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1175/2324] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 24 17:35:17 2022] POST /identity/v3/projects => generated 373 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:35:18.087189 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9b0361-9dab-4b59-b3a6-ef12f7af690e None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:18.098368 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9b0361-9dab-4b59-b3a6-ef12f7af690e None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:18.157318 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad1edbb-4dd9-4964-8367-d18bcf8d53cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:18.157967 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad1edbb-4dd9-4964-8367-d18bcf8d53cc admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:18.158300 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad1edbb-4dd9-4964-8367-d18bcf8d53cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:18.158571 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad1edbb-4dd9-4964-8367-d18bcf8d53cc admin admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:18.159248 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad1edbb-4dd9-4964-8367-d18bcf8d53cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:18.161375 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad1edbb-4dd9-4964-8367-d18bcf8d53cc admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:18.335251 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1150/2325] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 24 17:35:18 2022] POST /identity/v3/users => generated 293 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:35:18.336476 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71a33cb-d052-4ab9-9054-50fdbb2a58aa admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:18.353888 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71a33cb-d052-4ab9-9054-50fdbb2a58aa admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:18.361213 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3468f916-a90b-4e3f-b78c-0d51f495413d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:18.361836 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3468f916-a90b-4e3f-b78c-0d51f495413d admin admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:18.362112 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3468f916-a90b-4e3f-b78c-0d51f495413d admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:18.362387 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3468f916-a90b-4e3f-b78c-0d51f495413d admin admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:18.363224 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3468f916-a90b-4e3f-b78c-0d51f495413d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:18.364945 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3468f916-a90b-4e3f-b78c-0d51f495413d admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:18.376388 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1176/2326] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:35:18 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:35:18.386280 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad1edbb-4dd9-4964-8367-d18bcf8d53cc admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:18.391061 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad1edbb-4dd9-4964-8367-d18bcf8d53cc admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:18.396370 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb77aafc-918d-4c23-b35d-4829c642c132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:18.397086 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb77aafc-918d-4c23-b35d-4829c642c132 admin admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:18.397355 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb77aafc-918d-4c23-b35d-4829c642c132 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:18.397653 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb77aafc-918d-4c23-b35d-4829c642c132 admin admin] PATH_INFO: `/v3/projects/6942c8e041c94e1a83d1688851bd0fe4/users/1c739fda2d7e4281aebc311b05a64e08/roles/c5346bc54abf445f9900edf94264c850` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:18.413914 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb77aafc-918d-4c23-b35d-4829c642c132 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6942c8e041c94e1a83d1688851bd0fe4, user_id=1c739fda2d7e4281aebc311b05a64e08, role_id=c5346bc54abf445f9900edf94264c850)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:18.417217 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb77aafc-918d-4c23-b35d-4829c642c132 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:18.539291 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1151/2327] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 24 17:35:18 2022] PUT /identity/v3/projects/6942c8e041c94e1a83d1688851bd0fe4/users/1c739fda2d7e4281aebc311b05a64e08/roles/c5346bc54abf445f9900edf94264c850 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:35:18.544520 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3468f916-a90b-4e3f-b78c-0d51f495413d admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:18.556959 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3468f916-a90b-4e3f-b78c-0d51f495413d admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:18.625972 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6fa1a0-e920-4fc3-a968-265e03d535f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:18.626655 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6fa1a0-e920-4fc3-a968-265e03d535f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:18.627038 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6fa1a0-e920-4fc3-a968-265e03d535f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:18.627346 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6fa1a0-e920-4fc3-a968-265e03d535f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:18.628005 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6fa1a0-e920-4fc3-a968-265e03d535f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:18.629631 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6fa1a0-e920-4fc3-a968-265e03d535f3 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:18.642944 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1177/2328] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:35:18 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:35:18.645759 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb77aafc-918d-4c23-b35d-4829c642c132 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:18.657694 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb77aafc-918d-4c23-b35d-4829c642c132 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:18.662517 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951eb592-de38-43b7-8592-88ec19637835 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:18.663487 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951eb592-de38-43b7-8592-88ec19637835 admin admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:18.663637 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951eb592-de38-43b7-8592-88ec19637835 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:18.663843 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951eb592-de38-43b7-8592-88ec19637835 admin admin] PATH_INFO: `/v3/projects/6942c8e041c94e1a83d1688851bd0fe4/users/1c739fda2d7e4281aebc311b05a64e08/roles/c5346bc54abf445f9900edf94264c850` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:18.665947 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951eb592-de38-43b7-8592-88ec19637835 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6942c8e041c94e1a83d1688851bd0fe4, user_id=1c739fda2d7e4281aebc311b05a64e08, role_id=c5346bc54abf445f9900edf94264c850)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:18.668950 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951eb592-de38-43b7-8592-88ec19637835 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:18.680355 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1152/2329] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 24 17:35:18 2022] PUT /identity/v3/projects/6942c8e041c94e1a83d1688851bd0fe4/users/1c739fda2d7e4281aebc311b05a64e08/roles/c5346bc54abf445f9900edf94264c850 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:35:24.781035 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f53a15-612d-4c8f-8d57-b111ea590265 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:24.781908 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f53a15-612d-4c8f-8d57-b111ea590265 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:24.782204 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f53a15-612d-4c8f-8d57-b111ea590265 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:24.816081 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-f2f53a15-612d-4c8f-8d57-b111ea590265 None None] MFA Rules not processed for user `1c739fda2d7e4281aebc311b05a64e08`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:35:24.817698 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-f2f53a15-612d-4c8f-8d57-b111ea590265 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:35:24.868632 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:24.895653 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1178/2330] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:35:24 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:35:24.912851 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951eb592-de38-43b7-8592-88ec19637835 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:35:24.914759 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-951eb592-de38-43b7-8592-88ec19637835 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:35:25.045843 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951eb592-de38-43b7-8592-88ec19637835 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:35:25.053104 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20859b7-c58d-43ce-ac80-91e1d7cdb22a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557aca9255cb4bbb8af52e796c5c5177', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:35:25.053743 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20859b7-c58d-43ce-ac80-91e1d7cdb22a service nova] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:25.054026 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20859b7-c58d-43ce-ac80-91e1d7cdb22a service nova] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:25.054334 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20859b7-c58d-43ce-ac80-91e1d7cdb22a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:25.065044 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:25.121094 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20859b7-c58d-43ce-ac80-91e1d7cdb22a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:35:25.122631 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20859b7-c58d-43ce-ac80-91e1d7cdb22a service nova] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:35:25.132361 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1153/2331] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:35:24 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:35:28.409994 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acbfcda-5d10-42a5-b3df-d19168002064 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:35:28.410459 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acbfcda-5d10-42a5-b3df-d19168002064 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:35:28.410569 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acbfcda-5d10-42a5-b3df-d19168002064 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:35:28.432713 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-3acbfcda-5d10-42a5-b3df-d19168002064 None None] MFA Rules not processed for user `4df56db015514caa8aa92ffcfe227914`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:35:28.433917 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-3acbfcda-5d10-42a5-b3df-d19168002064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:35:28.441247 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1179/2332] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:35:28 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:35:35.655628 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=23 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:35:38.425635 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=12 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:13.833886 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20859b7-c58d-43ce-ac80-91e1d7cdb22a service nova] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:13.835403 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-f20859b7-c58d-43ce-ac80-91e1d7cdb22a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:36:13.898543 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20859b7-c58d-43ce-ac80-91e1d7cdb22a service nova] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:13.902689 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f489bf-2bc8-45d7-a2f5-1610ab6a1e5c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be06246d47584b529150ceb98a5cfd95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:13.903345 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f489bf-2bc8-45d7-a2f5-1610ab6a1e5c service cinder] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:13.903625 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f489bf-2bc8-45d7-a2f5-1610ab6a1e5c service cinder] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:13.903903 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f489bf-2bc8-45d7-a2f5-1610ab6a1e5c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:13.905754 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-97f489bf-2bc8-45d7-a2f5-1610ab6a1e5c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:36:13.919958 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f489bf-2bc8-45d7-a2f5-1610ab6a1e5c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:13.922940 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f489bf-2bc8-45d7-a2f5-1610ab6a1e5c service cinder] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:13.933129 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1154/2333] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:36:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:36:14.072738 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f090ac6-b575-4694-bd83-263200e04f5c None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:14.073235 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f090ac6-b575-4694-bd83-263200e04f5c None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:14.073317 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f090ac6-b575-4694-bd83-263200e04f5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:14.093593 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-0f090ac6-b575-4694-bd83-263200e04f5c None None] MFA Rules not processed for user `47da700d7d0f4d5cb1ea984a82d643bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:36:14.095055 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-0f090ac6-b575-4694-bd83-263200e04f5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:36:14.157398 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1180/2334] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:36:14 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:36:14.198843 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f489bf-2bc8-45d7-a2f5-1610ab6a1e5c service cinder] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:14.200289 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-97f489bf-2bc8-45d7-a2f5-1610ab6a1e5c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:36:14.211000 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f489bf-2bc8-45d7-a2f5-1610ab6a1e5c service cinder] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:14.214818 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff5f220-507c-4420-8710-69b8b99254cc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be06246d47584b529150ceb98a5cfd95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:14.221076 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff5f220-507c-4420-8710-69b8b99254cc service cinder] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:14.221076 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff5f220-507c-4420-8710-69b8b99254cc service cinder] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:14.221076 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff5f220-507c-4420-8710-69b8b99254cc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:14.221076 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-9ff5f220-507c-4420-8710-69b8b99254cc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:36:14.232516 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff5f220-507c-4420-8710-69b8b99254cc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:14.233951 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff5f220-507c-4420-8710-69b8b99254cc service cinder] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:14.246838 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1155/2335] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:36:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:36:19.340253 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f090ac6-b575-4694-bd83-263200e04f5c None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:19.341807 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-0f090ac6-b575-4694-bd83-263200e04f5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:36:19.422365 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f090ac6-b575-4694-bd83-263200e04f5c None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:19.427108 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4157baa-9faa-43fd-afcc-55d1db02133c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:19.427807 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4157baa-9faa-43fd-afcc-55d1db02133c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:19.428093 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4157baa-9faa-43fd-afcc-55d1db02133c admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:19.428293 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4157baa-9faa-43fd-afcc-55d1db02133c admin admin] PATH_INFO: `/v3/users/23413655643b47709304a632ec01d083` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:19.429857 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4157baa-9faa-43fd-afcc-55d1db02133c admin admin] RBAC: Authorizing `identity:delete_user(user_id=23413655643b47709304a632ec01d083)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:19.431800 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4157baa-9faa-43fd-afcc-55d1db02133c admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:19.636344 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-a4157baa-9faa-43fd-afcc-55d1db02133c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '23413655643b47709304a632ec01d083'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:36:19.659138 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-a4157baa-9faa-43fd-afcc-55d1db02133c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '23413655643b47709304a632ec01d083'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:36:19.773394 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-a4157baa-9faa-43fd-afcc-55d1db02133c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '23413655643b47709304a632ec01d083'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:36:19.774351 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-a4157baa-9faa-43fd-afcc-55d1db02133c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '23413655643b47709304a632ec01d083'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:36:19.786254 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-a4157baa-9faa-43fd-afcc-55d1db02133c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '23413655643b47709304a632ec01d083'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:36:19.794742 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1181/2336] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 24 17:36:19 2022] DELETE /identity/v3/users/23413655643b47709304a632ec01d083 => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:36:19.799676 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4460c7-8215-432b-8f81-5ffd7d947774 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:19.800214 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4460c7-8215-432b-8f81-5ffd7d947774 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:19.801070 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4460c7-8215-432b-8f81-5ffd7d947774 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:19.828060 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-7d4460c7-8215-432b-8f81-5ffd7d947774 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:36:19.829944 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-7d4460c7-8215-432b-8f81-5ffd7d947774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:36:19.940886 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1156/2337] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:36:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:36:19.963282 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4157baa-9faa-43fd-afcc-55d1db02133c admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:19.965150 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-a4157baa-9faa-43fd-afcc-55d1db02133c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:36:20.046391 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:20.085581 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4157baa-9faa-43fd-afcc-55d1db02133c admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:20.089445 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277471c7-8327-4f94-97da-29a4efd85fce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e9ed4d47aef4140bfb2d3bb798cc559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:20.090062 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277471c7-8327-4f94-97da-29a4efd85fce service neutron] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:20.090344 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277471c7-8327-4f94-97da-29a4efd85fce service neutron] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:20.090721 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277471c7-8327-4f94-97da-29a4efd85fce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:20.101525 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277471c7-8327-4f94-97da-29a4efd85fce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:20.103345 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277471c7-8327-4f94-97da-29a4efd85fce service neutron] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:20.118065 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1182/2338] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:36:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:36:21.022185 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4460c7-8215-432b-8f81-5ffd7d947774 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:21.027632 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-7d4460c7-8215-432b-8f81-5ffd7d947774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:36:21.029225 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277471c7-8327-4f94-97da-29a4efd85fce service neutron] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:21.032443 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-277471c7-8327-4f94-97da-29a4efd85fce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:36:21.053872 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4460c7-8215-432b-8f81-5ffd7d947774 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:21.067674 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdd0207-9ccb-4e45-b1dc-6e4fc76a68f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e9ed4d47aef4140bfb2d3bb798cc559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:21.070252 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdd0207-9ccb-4e45-b1dc-6e4fc76a68f8 service neutron] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:21.071539 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277471c7-8327-4f94-97da-29a4efd85fce service neutron] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:21.071961 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdd0207-9ccb-4e45-b1dc-6e4fc76a68f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:21.072379 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdd0207-9ccb-4e45-b1dc-6e4fc76a68f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:21.078525 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7333ea0e-bb65-4fc6-9b05-b4d2fd24abfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:21.079478 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7333ea0e-bb65-4fc6-9b05-b4d2fd24abfa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:21.079915 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7333ea0e-bb65-4fc6-9b05-b4d2fd24abfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:21.080372 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7333ea0e-bb65-4fc6-9b05-b4d2fd24abfa admin admin] PATH_INFO: `/v3/projects/39936f4fa2c04ddba1fa0f318dd69490` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:21.092288 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-6fdd0207-9ccb-4e45-b1dc-6e4fc76a68f8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:36:21.096450 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7333ea0e-bb65-4fc6-9b05-b4d2fd24abfa admin admin] RBAC: Authorizing `identity:delete_project(project_id=39936f4fa2c04ddba1fa0f318dd69490)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:21.103573 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7333ea0e-bb65-4fc6-9b05-b4d2fd24abfa admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:21.116716 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdd0207-9ccb-4e45-b1dc-6e4fc76a68f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:21.120404 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdd0207-9ccb-4e45-b1dc-6e4fc76a68f8 service neutron] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:21.133214 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1157/2339] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:36:21 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:36:21.281506 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-7333ea0e-bb65-4fc6-9b05-b4d2fd24abfa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39936f4fa2c04ddba1fa0f318dd69490', 'request_id': 'req-7333ea0e-bb65-4fc6-9b05-b4d2fd24abfa'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:36:21.435732 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-7333ea0e-bb65-4fc6-9b05-b4d2fd24abfa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39936f4fa2c04ddba1fa0f318dd69490', 'request_id': 'req-7333ea0e-bb65-4fc6-9b05-b4d2fd24abfa'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:36:21.446016 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-7333ea0e-bb65-4fc6-9b05-b4d2fd24abfa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39936f4fa2c04ddba1fa0f318dd69490', 'request_id': 'req-7333ea0e-bb65-4fc6-9b05-b4d2fd24abfa'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:36:21.468644 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-7333ea0e-bb65-4fc6-9b05-b4d2fd24abfa admin admin] The token cache is being invalidate because project 39936f4fa2c04ddba1fa0f318dd69490 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81731) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:36:21.469243 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-7333ea0e-bb65-4fc6-9b05-b4d2fd24abfa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:36:21.471073 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-7333ea0e-bb65-4fc6-9b05-b4d2fd24abfa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:36:21.475891 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1183/2340] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 24 17:36:21 2022] DELETE /identity/v3/projects/39936f4fa2c04ddba1fa0f318dd69490 => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:36:21.479935 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdd0207-9ccb-4e45-b1dc-6e4fc76a68f8 service neutron] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:21.492898 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdd0207-9ccb-4e45-b1dc-6e4fc76a68f8 service neutron] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:21.577762 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9355c1-bf05-40ac-b243-946104e61c5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:21.578410 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9355c1-bf05-40ac-b243-946104e61c5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:21.578877 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9355c1-bf05-40ac-b243-946104e61c5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:21.579190 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9355c1-bf05-40ac-b243-946104e61c5f admin admin] PATH_INFO: `/v3/users/f4f6c5d0bd2946a2b7b5c5c4988b9bf3` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:21.580382 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9355c1-bf05-40ac-b243-946104e61c5f admin admin] RBAC: Authorizing `identity:delete_user(user_id=f4f6c5d0bd2946a2b7b5c5c4988b9bf3)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:21.583007 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9355c1-bf05-40ac-b243-946104e61c5f admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:21.616332 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:21.826949 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-0b9355c1-bf05-40ac-b243-946104e61c5f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f4f6c5d0bd2946a2b7b5c5c4988b9bf3'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:36:21.945211 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-0b9355c1-bf05-40ac-b243-946104e61c5f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f4f6c5d0bd2946a2b7b5c5c4988b9bf3'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:36:21.946473 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-0b9355c1-bf05-40ac-b243-946104e61c5f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f4f6c5d0bd2946a2b7b5c5c4988b9bf3'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:36:21.961378 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-0b9355c1-bf05-40ac-b243-946104e61c5f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f4f6c5d0bd2946a2b7b5c5c4988b9bf3'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:36:21.962556 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-0b9355c1-bf05-40ac-b243-946104e61c5f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f4f6c5d0bd2946a2b7b5c5c4988b9bf3'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:36:21.988680 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1158/2341] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 24 17:36:21 2022] DELETE /identity/v3/users/f4f6c5d0bd2946a2b7b5c5c4988b9bf3 => generated 0 bytes in 509 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:36:22.740951 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7333ea0e-bb65-4fc6-9b05-b4d2fd24abfa admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:22.742441 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-7333ea0e-bb65-4fc6-9b05-b4d2fd24abfa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:36:22.758341 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7333ea0e-bb65-4fc6-9b05-b4d2fd24abfa admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:22.844092 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c32386-b7b1-4ece-aac1-1130fccd4e9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:22.844643 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c32386-b7b1-4ece-aac1-1130fccd4e9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:22.844927 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c32386-b7b1-4ece-aac1-1130fccd4e9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:22.845217 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c32386-b7b1-4ece-aac1-1130fccd4e9d admin admin] PATH_INFO: `/v3/projects/10dbc3f9674b4190be573f991de51cec` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:22.846387 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c32386-b7b1-4ece-aac1-1130fccd4e9d admin admin] RBAC: Authorizing `identity:delete_project(project_id=10dbc3f9674b4190be573f991de51cec)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:22.848970 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c32386-b7b1-4ece-aac1-1130fccd4e9d admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:22.986257 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-94c32386-b7b1-4ece-aac1-1130fccd4e9d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '10dbc3f9674b4190be573f991de51cec', 'request_id': 'req-94c32386-b7b1-4ece-aac1-1130fccd4e9d'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:36:22.988860 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:23.081065 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-94c32386-b7b1-4ece-aac1-1130fccd4e9d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '10dbc3f9674b4190be573f991de51cec', 'request_id': 'req-94c32386-b7b1-4ece-aac1-1130fccd4e9d'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:36:23.088947 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-94c32386-b7b1-4ece-aac1-1130fccd4e9d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '10dbc3f9674b4190be573f991de51cec', 'request_id': 'req-94c32386-b7b1-4ece-aac1-1130fccd4e9d'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:36:23.101729 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-94c32386-b7b1-4ece-aac1-1130fccd4e9d admin admin] The token cache is being invalidate because project 10dbc3f9674b4190be573f991de51cec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81731) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:36:23.101964 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-94c32386-b7b1-4ece-aac1-1130fccd4e9d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:36:23.102819 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-94c32386-b7b1-4ece-aac1-1130fccd4e9d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:36:23.105101 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1184/2342] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 24 17:36:22 2022] DELETE /identity/v3/projects/10dbc3f9674b4190be573f991de51cec => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:36:23.112591 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f9422a-b6a0-4d9f-b7b2-6f5c48f674ea None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:23.113219 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f9422a-b6a0-4d9f-b7b2-6f5c48f674ea None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:23.113219 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f9422a-b6a0-4d9f-b7b2-6f5c48f674ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:23.135461 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-87f9422a-b6a0-4d9f-b7b2-6f5c48f674ea None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:36:23.136671 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-87f9422a-b6a0-4d9f-b7b2-6f5c48f674ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:36:23.201471 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1159/2343] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:36:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:36:23.210020 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259c612b-d6de-483d-8885-1f896a310163 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:23.210299 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259c612b-d6de-483d-8885-1f896a310163 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:23.210576 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259c612b-d6de-483d-8885-1f896a310163 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:23.230184 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-259c612b-d6de-483d-8885-1f896a310163 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:36:23.231463 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-259c612b-d6de-483d-8885-1f896a310163 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:36:23.239807 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1185/2344] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:36:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:36:23.251815 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f9422a-b6a0-4d9f-b7b2-6f5c48f674ea None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:23.263026 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f9422a-b6a0-4d9f-b7b2-6f5c48f674ea None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:23.270252 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8492b5bc-1a50-480b-bc32-b3575813d0e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:23.271349 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8492b5bc-1a50-480b-bc32-b3575813d0e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:23.271701 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8492b5bc-1a50-480b-bc32-b3575813d0e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:23.271994 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8492b5bc-1a50-480b-bc32-b3575813d0e0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:23.272631 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8492b5bc-1a50-480b-bc32-b3575813d0e0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:36:23.272930 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8492b5bc-1a50-480b-bc32-b3575813d0e0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:23.274574 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8492b5bc-1a50-480b-bc32-b3575813d0e0 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:23.288312 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1160/2345] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 24 17:36:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:36:23.293265 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259c612b-d6de-483d-8885-1f896a310163 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:23.302692 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259c612b-d6de-483d-8885-1f896a310163 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:23.307788 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6a8af6-28eb-4c7e-91d6-74e2e8773a30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:23.308421 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6a8af6-28eb-4c7e-91d6-74e2e8773a30 admin admin] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:23.308838 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6a8af6-28eb-4c7e-91d6-74e2e8773a30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:23.309017 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6a8af6-28eb-4c7e-91d6-74e2e8773a30 admin admin] PATH_INFO: `/v3/projects` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:23.309648 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6a8af6-28eb-4c7e-91d6-74e2e8773a30 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:23.311592 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6a8af6-28eb-4c7e-91d6-74e2e8773a30 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:23.466334 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1186/2346] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 24 17:36:23 2022] POST /identity/v3/projects => generated 357 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:36:23.470791 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8492b5bc-1a50-480b-bc32-b3575813d0e0 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:23.483705 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8492b5bc-1a50-480b-bc32-b3575813d0e0 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:23.557887 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bef2d2-a80e-4838-9b4d-c78eaf47209d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:23.558512 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bef2d2-a80e-4838-9b4d-c78eaf47209d admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:23.558795 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bef2d2-a80e-4838-9b4d-c78eaf47209d admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:23.559105 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bef2d2-a80e-4838-9b4d-c78eaf47209d admin admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:23.559747 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bef2d2-a80e-4838-9b4d-c78eaf47209d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:23.561729 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bef2d2-a80e-4838-9b4d-c78eaf47209d admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:23.670762 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1161/2347] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 24 17:36:23 2022] POST /identity/v3/users => generated 279 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:36:23.676481 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6a8af6-28eb-4c7e-91d6-74e2e8773a30 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:23.688639 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6a8af6-28eb-4c7e-91d6-74e2e8773a30 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:23.693515 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03a9c6f-fad5-4d0b-8dd9-12d1a92070d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:23.694131 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03a9c6f-fad5-4d0b-8dd9-12d1a92070d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:23.694411 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03a9c6f-fad5-4d0b-8dd9-12d1a92070d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:23.694681 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03a9c6f-fad5-4d0b-8dd9-12d1a92070d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:23.695531 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03a9c6f-fad5-4d0b-8dd9-12d1a92070d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:23.697119 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03a9c6f-fad5-4d0b-8dd9-12d1a92070d2 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:23.708071 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1187/2348] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:36:23 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:36:23.712167 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bef2d2-a80e-4838-9b4d-c78eaf47209d admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:23.728301 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bef2d2-a80e-4838-9b4d-c78eaf47209d admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:23.733831 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2594ee-2ce7-4fb5-bad7-d9761c68c32b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:23.734451 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2594ee-2ce7-4fb5-bad7-d9761c68c32b admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:23.734726 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2594ee-2ce7-4fb5-bad7-d9761c68c32b admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:23.735030 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2594ee-2ce7-4fb5-bad7-d9761c68c32b admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:23.735674 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2594ee-2ce7-4fb5-bad7-d9761c68c32b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:23.737303 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2594ee-2ce7-4fb5-bad7-d9761c68c32b admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:23.754715 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1162/2349] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:36:23 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:36:23.759397 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03a9c6f-fad5-4d0b-8dd9-12d1a92070d2 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:23.770501 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03a9c6f-fad5-4d0b-8dd9-12d1a92070d2 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:23.776638 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2513312d-377c-413f-8006-1386999a6401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:23.777248 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2513312d-377c-413f-8006-1386999a6401 admin admin] REQUEST_METHOD: `PUT` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:23.777527 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2513312d-377c-413f-8006-1386999a6401 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:23.777796 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2513312d-377c-413f-8006-1386999a6401 admin admin] PATH_INFO: `/v3/projects/9a9b468c4b83453f80969ee6b488c934/users/bfc4cf21e71c4d48a95dc9faf19136d3/roles/60dc7e16aee746d5a3e8aa27d7d8525c` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:23.793831 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2513312d-377c-413f-8006-1386999a6401 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a9b468c4b83453f80969ee6b488c934, user_id=bfc4cf21e71c4d48a95dc9faf19136d3, role_id=60dc7e16aee746d5a3e8aa27d7d8525c)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:23.796777 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2513312d-377c-413f-8006-1386999a6401 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:23.882816 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1188/2350] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 24 17:36:23 2022] PUT /identity/v3/projects/9a9b468c4b83453f80969ee6b488c934/users/bfc4cf21e71c4d48a95dc9faf19136d3/roles/60dc7e16aee746d5a3e8aa27d7d8525c => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:36:23.894561 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cf9f6c-6ab6-4bc5-823f-7434ce29345a None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:23.894851 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cf9f6c-6ab6-4bc5-823f-7434ce29345a None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:23.895166 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cf9f6c-6ab6-4bc5-823f-7434ce29345a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:23.902964 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:DELETE=8,keystone:SELECT=88,keystone:INSERT=4 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:23.930678 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-80cf9f6c-6ab6-4bc5-823f-7434ce29345a None None] MFA Rules not processed for user `bfc4cf21e71c4d48a95dc9faf19136d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:36:23.931836 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-80cf9f6c-6ab6-4bc5-823f-7434ce29345a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:36:24.024426 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1163/2351] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:36:23 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:36:24.025915 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2513312d-377c-413f-8006-1386999a6401 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:24.111830 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2513312d-377c-413f-8006-1386999a6401 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:24.117435 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3006680-e4dc-4c2a-8244-397152ecd9db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:24.118057 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3006680-e4dc-4c2a-8244-397152ecd9db admin admin] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:24.118349 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3006680-e4dc-4c2a-8244-397152ecd9db admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:24.118689 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3006680-e4dc-4c2a-8244-397152ecd9db admin admin] PATH_INFO: `/v3/projects` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:24.119349 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3006680-e4dc-4c2a-8244-397152ecd9db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:24.121249 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3006680-e4dc-4c2a-8244-397152ecd9db admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:24.357706 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1189/2352] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 24 17:36:24 2022] POST /identity/v3/projects => generated 357 bytes in 332 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:36:24.362353 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cf9f6c-6ab6-4bc5-823f-7434ce29345a None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:24.373892 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cf9f6c-6ab6-4bc5-823f-7434ce29345a None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:24.430889 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768492da-8aeb-49d7-ac85-39c3e30d37a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:24.431526 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768492da-8aeb-49d7-ac85-39c3e30d37a0 admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:24.431802 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768492da-8aeb-49d7-ac85-39c3e30d37a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:24.432072 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768492da-8aeb-49d7-ac85-39c3e30d37a0 admin admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:24.432688 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768492da-8aeb-49d7-ac85-39c3e30d37a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:24.434757 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768492da-8aeb-49d7-ac85-39c3e30d37a0 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:24.466109 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3006680-e4dc-4c2a-8244-397152ecd9db admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:24.468467 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-e3006680-e4dc-4c2a-8244-397152ecd9db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:36:24.528174 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:24.552306 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3006680-e4dc-4c2a-8244-397152ecd9db admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:24.558039 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51394df1-2c88-4c15-97f5-d9ca89333934 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557aca9255cb4bbb8af52e796c5c5177', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:24.565667 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51394df1-2c88-4c15-97f5-d9ca89333934 service nova] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:24.565667 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51394df1-2c88-4c15-97f5-d9ca89333934 service nova] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:24.565667 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51394df1-2c88-4c15-97f5-d9ca89333934 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:24.565667 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-51394df1-2c88-4c15-97f5-d9ca89333934 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:36:24.595751 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1164/2353] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 24 17:36:24 2022] POST /identity/v3/users => generated 285 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:36:24.600430 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768492da-8aeb-49d7-ac85-39c3e30d37a0 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:24.615315 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768492da-8aeb-49d7-ac85-39c3e30d37a0 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:24.616784 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f944c6b-a319-40c5-9400-b25cb21212b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:24.617396 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f944c6b-a319-40c5-9400-b25cb21212b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:24.617671 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f944c6b-a319-40c5-9400-b25cb21212b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:24.617943 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f944c6b-a319-40c5-9400-b25cb21212b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:24.618598 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f944c6b-a319-40c5-9400-b25cb21212b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:24.620304 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f944c6b-a319-40c5-9400-b25cb21212b6 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:24.632569 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1165/2354] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:36:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:36:24.637137 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f944c6b-a319-40c5-9400-b25cb21212b6 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:24.649620 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f944c6b-a319-40c5-9400-b25cb21212b6 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:24.654762 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba1f71f-82ac-4ca1-86be-f9918f741e4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:24.655459 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba1f71f-82ac-4ca1-86be-f9918f741e4a admin admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:24.655736 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba1f71f-82ac-4ca1-86be-f9918f741e4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:24.656011 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba1f71f-82ac-4ca1-86be-f9918f741e4a admin admin] PATH_INFO: `/v3/projects/7394ae5eaff64d52a5d4f49fb4cb58d1/users/999702f3c396486da01d99726a59a9fb/roles/c5346bc54abf445f9900edf94264c850` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:24.672015 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba1f71f-82ac-4ca1-86be-f9918f741e4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=7394ae5eaff64d52a5d4f49fb4cb58d1, user_id=999702f3c396486da01d99726a59a9fb, role_id=c5346bc54abf445f9900edf94264c850)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:24.674633 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba1f71f-82ac-4ca1-86be-f9918f741e4a admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:24.681103 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51394df1-2c88-4c15-97f5-d9ca89333934 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:24.683620 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51394df1-2c88-4c15-97f5-d9ca89333934 service nova] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:24.705792 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1190/2355] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:36:24 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:36:24.748062 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1166/2356] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 24 17:36:24 2022] PUT /identity/v3/projects/7394ae5eaff64d52a5d4f49fb4cb58d1/users/999702f3c396486da01d99726a59a9fb/roles/c5346bc54abf445f9900edf94264c850 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:36:24.755272 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51394df1-2c88-4c15-97f5-d9ca89333934 service nova] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:24.770517 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51394df1-2c88-4c15-97f5-d9ca89333934 service nova] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:24.855193 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ea318a-3d01-446a-bdfd-5144d1b2ac1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:24.856341 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ea318a-3d01-446a-bdfd-5144d1b2ac1a admin admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:24.856811 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ea318a-3d01-446a-bdfd-5144d1b2ac1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:24.858244 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ea318a-3d01-446a-bdfd-5144d1b2ac1a admin admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:24.859566 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ea318a-3d01-446a-bdfd-5144d1b2ac1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:24.860749 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ea318a-3d01-446a-bdfd-5144d1b2ac1a admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:24.876761 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1191/2357] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:36:24 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:36:24.882580 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba1f71f-82ac-4ca1-86be-f9918f741e4a admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:24.902475 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba1f71f-82ac-4ca1-86be-f9918f741e4a admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:24.911439 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dcdbe6-f1ea-4ce4-9b97-71f690d95afb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:24.912267 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dcdbe6-f1ea-4ce4-9b97-71f690d95afb admin admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:24.912597 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dcdbe6-f1ea-4ce4-9b97-71f690d95afb admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:24.912873 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dcdbe6-f1ea-4ce4-9b97-71f690d95afb admin admin] PATH_INFO: `/v3/projects/7394ae5eaff64d52a5d4f49fb4cb58d1/users/999702f3c396486da01d99726a59a9fb/roles/c5346bc54abf445f9900edf94264c850` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:24.916013 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dcdbe6-f1ea-4ce4-9b97-71f690d95afb admin admin] RBAC: Authorizing `identity:create_grant(project_id=7394ae5eaff64d52a5d4f49fb4cb58d1, user_id=999702f3c396486da01d99726a59a9fb, role_id=c5346bc54abf445f9900edf94264c850)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:24.921343 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dcdbe6-f1ea-4ce4-9b97-71f690d95afb admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:24.936750 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1167/2358] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 24 17:36:24 2022] PUT /identity/v3/projects/7394ae5eaff64d52a5d4f49fb4cb58d1/users/999702f3c396486da01d99726a59a9fb/roles/c5346bc54abf445f9900edf94264c850 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:36:24.950560 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95de05ef-7a9f-406a-83e1-5924dac9b1eb None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:24.950850 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95de05ef-7a9f-406a-83e1-5924dac9b1eb None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:24.951246 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95de05ef-7a9f-406a-83e1-5924dac9b1eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:24.989492 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-95de05ef-7a9f-406a-83e1-5924dac9b1eb None None] MFA Rules not processed for user `999702f3c396486da01d99726a59a9fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:36:24.991146 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-95de05ef-7a9f-406a-83e1-5924dac9b1eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:36:25.045674 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:25.074050 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1192/2359] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:36:24 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:36:25.086645 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dcdbe6-f1ea-4ce4-9b97-71f690d95afb admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:25.088458 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-42dcdbe6-f1ea-4ce4-9b97-71f690d95afb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:36:25.133337 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:25.169306 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dcdbe6-f1ea-4ce4-9b97-71f690d95afb admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:25.176575 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d20976e-db15-439f-bcdd-9fdd69e208c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be06246d47584b529150ceb98a5cfd95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:25.177254 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d20976e-db15-439f-bcdd-9fdd69e208c1 service cinder] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:25.177563 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d20976e-db15-439f-bcdd-9fdd69e208c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:25.178031 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d20976e-db15-439f-bcdd-9fdd69e208c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:25.192028 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d20976e-db15-439f-bcdd-9fdd69e208c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:25.193603 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d20976e-db15-439f-bcdd-9fdd69e208c1 service cinder] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:25.213481 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1168/2360] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:36:25 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:36:25.506559 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95de05ef-7a9f-406a-83e1-5924dac9b1eb None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:25.522474 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95de05ef-7a9f-406a-83e1-5924dac9b1eb None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:25.529874 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d200a4-8b25-4843-8256-2a5cb055feb9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be06246d47584b529150ceb98a5cfd95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:25.530698 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d200a4-8b25-4843-8256-2a5cb055feb9 service cinder] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:25.531226 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d200a4-8b25-4843-8256-2a5cb055feb9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:25.531695 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d200a4-8b25-4843-8256-2a5cb055feb9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:25.620992 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d200a4-8b25-4843-8256-2a5cb055feb9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:25.623147 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d200a4-8b25-4843-8256-2a5cb055feb9 service cinder] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:25.633407 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1193/2361] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:36:25 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:36:35.296440 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=11 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:35.628181 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=36 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:38.221399 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d20976e-db15-439f-bcdd-9fdd69e208c1 service cinder] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:38.321029 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d20976e-db15-439f-bcdd-9fdd69e208c1 service cinder] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:38.335739 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709f503d-1c5d-478d-9f44-bd37e591f547 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557aca9255cb4bbb8af52e796c5c5177', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:38.339743 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709f503d-1c5d-478d-9f44-bd37e591f547 service nova] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:38.339743 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709f503d-1c5d-478d-9f44-bd37e591f547 service nova] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:38.339743 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709f503d-1c5d-478d-9f44-bd37e591f547 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:38.339743 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-709f503d-1c5d-478d-9f44-bd37e591f547 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:36:38.378204 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709f503d-1c5d-478d-9f44-bd37e591f547 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:38.388430 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709f503d-1c5d-478d-9f44-bd37e591f547 service nova] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:38.402945 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1169/2362] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:36:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:36:40.520895 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d200a4-8b25-4843-8256-2a5cb055feb9 service cinder] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:40.522376 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-f2d200a4-8b25-4843-8256-2a5cb055feb9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:36:40.605399 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d200a4-8b25-4843-8256-2a5cb055feb9 service cinder] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:40.610431 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c666415d-0042-459d-9ed6-9dbb0e875809 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e9ed4d47aef4140bfb2d3bb798cc559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:40.611250 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c666415d-0042-459d-9ed6-9dbb0e875809 service neutron] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:40.611533 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c666415d-0042-459d-9ed6-9dbb0e875809 service neutron] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:40.611875 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c666415d-0042-459d-9ed6-9dbb0e875809 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:40.613609 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-c666415d-0042-459d-9ed6-9dbb0e875809 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:36:40.682829 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c666415d-0042-459d-9ed6-9dbb0e875809 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:40.684332 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c666415d-0042-459d-9ed6-9dbb0e875809 service neutron] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:40.697001 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1194/2363] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:36:40 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:36:42.955577 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709f503d-1c5d-478d-9f44-bd37e591f547 service nova] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:42.974073 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709f503d-1c5d-478d-9f44-bd37e591f547 service nova] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:42.981141 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e189c2b-d6d2-4c7c-a798-34ad260fcde0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557aca9255cb4bbb8af52e796c5c5177', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:42.981900 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e189c2b-d6d2-4c7c-a798-34ad260fcde0 service nova] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:42.982183 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e189c2b-d6d2-4c7c-a798-34ad260fcde0 service nova] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:42.982457 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e189c2b-d6d2-4c7c-a798-34ad260fcde0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:42.984415 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-6e189c2b-d6d2-4c7c-a798-34ad260fcde0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:36:43.115438 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e189c2b-d6d2-4c7c-a798-34ad260fcde0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:43.116985 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e189c2b-d6d2-4c7c-a798-34ad260fcde0 service nova] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:43.140783 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1170/2364] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:36:42 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:36:43.606516 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c666415d-0042-459d-9ed6-9dbb0e875809 service neutron] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:43.609331 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-c666415d-0042-459d-9ed6-9dbb0e875809 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:36:43.635349 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c666415d-0042-459d-9ed6-9dbb0e875809 service neutron] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:43.645551 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb04416-9f6c-410a-b0d1-e0f32b819ad3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557aca9255cb4bbb8af52e796c5c5177', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:43.646437 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb04416-9f6c-410a-b0d1-e0f32b819ad3 service nova] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:43.647335 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb04416-9f6c-410a-b0d1-e0f32b819ad3 service nova] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:43.649027 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb04416-9f6c-410a-b0d1-e0f32b819ad3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:43.651981 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-efb04416-9f6c-410a-b0d1-e0f32b819ad3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:36:43.681308 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb04416-9f6c-410a-b0d1-e0f32b819ad3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:43.684761 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb04416-9f6c-410a-b0d1-e0f32b819ad3 service nova] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:43.704518 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1195/2365] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:36:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:36:43.730362 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e189c2b-d6d2-4c7c-a798-34ad260fcde0 service nova] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:43.732236 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-6e189c2b-d6d2-4c7c-a798-34ad260fcde0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:36:43.745178 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e189c2b-d6d2-4c7c-a798-34ad260fcde0 service nova] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:43.749214 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7d5abe-d2b7-4f04-b894-d61aa4ff4050 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e9ed4d47aef4140bfb2d3bb798cc559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:43.749931 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7d5abe-d2b7-4f04-b894-d61aa4ff4050 service neutron] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:43.750307 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7d5abe-d2b7-4f04-b894-d61aa4ff4050 service neutron] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:43.750649 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7d5abe-d2b7-4f04-b894-d61aa4ff4050 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:43.752834 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-3d7d5abe-d2b7-4f04-b894-d61aa4ff4050 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:36:43.839246 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:43.871252 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7d5abe-d2b7-4f04-b894-d61aa4ff4050 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:43.876165 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7d5abe-d2b7-4f04-b894-d61aa4ff4050 service neutron] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:43.901456 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1171/2366] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:36:43 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:36:51.531908 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb04416-9f6c-410a-b0d1-e0f32b819ad3 service nova] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:51.610290 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb04416-9f6c-410a-b0d1-e0f32b819ad3 service nova] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:51.616867 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3b4c44-acd5-47cc-9bb0-ff6acc1e0d91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:51.617695 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3b4c44-acd5-47cc-9bb0-ff6acc1e0d91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:51.618120 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3b4c44-acd5-47cc-9bb0-ff6acc1e0d91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:51.618526 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3b4c44-acd5-47cc-9bb0-ff6acc1e0d91 admin admin] PATH_INFO: `/v3/users/bfc4cf21e71c4d48a95dc9faf19136d3` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:51.619952 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3b4c44-acd5-47cc-9bb0-ff6acc1e0d91 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bfc4cf21e71c4d48a95dc9faf19136d3)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:51.622404 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3b4c44-acd5-47cc-9bb0-ff6acc1e0d91 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:51.865209 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:51.871583 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-5b3b4c44-acd5-47cc-9bb0-ff6acc1e0d91 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bfc4cf21e71c4d48a95dc9faf19136d3'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:36:51.886202 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-5b3b4c44-acd5-47cc-9bb0-ff6acc1e0d91 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bfc4cf21e71c4d48a95dc9faf19136d3'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:36:51.982638 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-5b3b4c44-acd5-47cc-9bb0-ff6acc1e0d91 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bfc4cf21e71c4d48a95dc9faf19136d3'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:36:51.983556 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-5b3b4c44-acd5-47cc-9bb0-ff6acc1e0d91 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bfc4cf21e71c4d48a95dc9faf19136d3'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:36:51.987908 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17a95f0-b415-4720-9aac-057defebaaf2 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:51.988414 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17a95f0-b415-4720-9aac-057defebaaf2 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:51.988916 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17a95f0-b415-4720-9aac-057defebaaf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:51.993582 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-5b3b4c44-acd5-47cc-9bb0-ff6acc1e0d91 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bfc4cf21e71c4d48a95dc9faf19136d3'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:36:52.000775 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1196/2367] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 24 17:36:51 2022] DELETE /identity/v3/users/bfc4cf21e71c4d48a95dc9faf19136d3 => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:36:52.005791 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64103a9f-2d3a-44c0-b668-e6bc6493a778 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:52.006092 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64103a9f-2d3a-44c0-b668-e6bc6493a778 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:52.006359 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64103a9f-2d3a-44c0-b668-e6bc6493a778 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:52.026241 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-64103a9f-2d3a-44c0-b668-e6bc6493a778 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:36:52.027649 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-64103a9f-2d3a-44c0-b668-e6bc6493a778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:36:52.048518 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-e17a95f0-b415-4720-9aac-057defebaaf2 None None] MFA Rules not processed for user `0ccc6723f7514e52863d102de3839951`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:36:52.049744 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-e17a95f0-b415-4720-9aac-057defebaaf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:36:52.104786 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1197/2368] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:36:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:36:52.115539 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64103a9f-2d3a-44c0-b668-e6bc6493a778 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:52.117042 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-64103a9f-2d3a-44c0-b668-e6bc6493a778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:36:52.122229 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1172/2369] 10.222.0.18 () {58 vars in 1009 bytes} [Wed Aug 24 17:36:51 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:36:52.179582 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64103a9f-2d3a-44c0-b668-e6bc6493a778 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:52.183669 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a12863b-cfb9-4e7d-9413-2bb856ee2473 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e9ed4d47aef4140bfb2d3bb798cc559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:52.184273 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a12863b-cfb9-4e7d-9413-2bb856ee2473 service neutron] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:52.184555 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a12863b-cfb9-4e7d-9413-2bb856ee2473 service neutron] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:52.184875 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a12863b-cfb9-4e7d-9413-2bb856ee2473 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:52.194879 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a12863b-cfb9-4e7d-9413-2bb856ee2473 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:52.196340 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a12863b-cfb9-4e7d-9413-2bb856ee2473 service neutron] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:52.206082 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1198/2370] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:36:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:36:52.283486 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17a95f0-b415-4720-9aac-057defebaaf2 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:52.285829 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-e17a95f0-b415-4720-9aac-057defebaaf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:36:52.390811 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17a95f0-b415-4720-9aac-057defebaaf2 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:52.396821 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80f0507-c351-462b-817f-73c05092f8a4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef8caa353be34499a58cd25b72ee4032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:52.397465 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80f0507-c351-462b-817f-73c05092f8a4 service swift] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:52.397830 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80f0507-c351-462b-817f-73c05092f8a4 service swift] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:52.398108 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80f0507-c351-462b-817f-73c05092f8a4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:52.407150 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80f0507-c351-462b-817f-73c05092f8a4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:52.408774 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80f0507-c351-462b-817f-73c05092f8a4 service swift] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:52.419109 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1173/2371] 10.222.0.18 () {60 vars in 1384 bytes} [Wed Aug 24 17:36:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 17:36:52.727078 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a12863b-cfb9-4e7d-9413-2bb856ee2473 service neutron] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:52.729477 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-0a12863b-cfb9-4e7d-9413-2bb856ee2473 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:36:52.763258 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a12863b-cfb9-4e7d-9413-2bb856ee2473 service neutron] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:52.773117 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4083fb-1340-4dce-b54d-16d040830daf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:52.773350 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4083fb-1340-4dce-b54d-16d040830daf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:52.773703 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4083fb-1340-4dce-b54d-16d040830daf admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:52.774221 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4083fb-1340-4dce-b54d-16d040830daf admin admin] PATH_INFO: `/v3/projects/9a9b468c4b83453f80969ee6b488c934` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:52.775725 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4083fb-1340-4dce-b54d-16d040830daf admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a9b468c4b83453f80969ee6b488c934)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:52.778978 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4083fb-1340-4dce-b54d-16d040830daf admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:53.007368 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-eb4083fb-1340-4dce-b54d-16d040830daf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a9b468c4b83453f80969ee6b488c934', 'request_id': 'req-eb4083fb-1340-4dce-b54d-16d040830daf'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:36:53.013427 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:DELETE=10,keystone:SELECT=88,keystone:INSERT=2 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:53.148414 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-eb4083fb-1340-4dce-b54d-16d040830daf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a9b468c4b83453f80969ee6b488c934', 'request_id': 'req-eb4083fb-1340-4dce-b54d-16d040830daf'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:36:53.166340 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-eb4083fb-1340-4dce-b54d-16d040830daf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a9b468c4b83453f80969ee6b488c934', 'request_id': 'req-eb4083fb-1340-4dce-b54d-16d040830daf'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:36:53.200005 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-eb4083fb-1340-4dce-b54d-16d040830daf admin admin] The token cache is being invalidate because project 9a9b468c4b83453f80969ee6b488c934 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81731) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:36:53.200295 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-eb4083fb-1340-4dce-b54d-16d040830daf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:36:53.200959 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-eb4083fb-1340-4dce-b54d-16d040830daf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:36:53.202451 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1199/2372] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 24 17:36:52 2022] DELETE /identity/v3/projects/9a9b468c4b83453f80969ee6b488c934 => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:36:53.207582 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80f0507-c351-462b-817f-73c05092f8a4 service swift] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:53.220631 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80f0507-c351-462b-817f-73c05092f8a4 service swift] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:53.257572 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:53.298172 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7a4fea-931c-4194-9279-59980ccaffea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:53.299340 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7a4fea-931c-4194-9279-59980ccaffea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:53.299589 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7a4fea-931c-4194-9279-59980ccaffea admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:53.299870 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7a4fea-931c-4194-9279-59980ccaffea admin admin] PATH_INFO: `/v3/users/999702f3c396486da01d99726a59a9fb` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:53.301608 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7a4fea-931c-4194-9279-59980ccaffea admin admin] RBAC: Authorizing `identity:delete_user(user_id=999702f3c396486da01d99726a59a9fb)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:53.303963 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7a4fea-931c-4194-9279-59980ccaffea admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:53.597126 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-af7a4fea-931c-4194-9279-59980ccaffea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '999702f3c396486da01d99726a59a9fb'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:36:53.744378 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-af7a4fea-931c-4194-9279-59980ccaffea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '999702f3c396486da01d99726a59a9fb'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:36:53.745978 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-af7a4fea-931c-4194-9279-59980ccaffea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '999702f3c396486da01d99726a59a9fb'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:36:53.765912 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-af7a4fea-931c-4194-9279-59980ccaffea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '999702f3c396486da01d99726a59a9fb'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:36:53.767350 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-af7a4fea-931c-4194-9279-59980ccaffea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '999702f3c396486da01d99726a59a9fb'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:36:53.815564 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1174/2373] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 24 17:36:53 2022] DELETE /identity/v3/users/999702f3c396486da01d99726a59a9fb => generated 0 bytes in 609 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:36:54.543978 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4083fb-1340-4dce-b54d-16d040830daf admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:54.545588 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-eb4083fb-1340-4dce-b54d-16d040830daf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:36:54.576746 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4083fb-1340-4dce-b54d-16d040830daf admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:54.685725 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fdcfd3-cecc-46d3-bf19-c563e2e95d07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:54.686289 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fdcfd3-cecc-46d3-bf19-c563e2e95d07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:54.686576 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fdcfd3-cecc-46d3-bf19-c563e2e95d07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:54.687019 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fdcfd3-cecc-46d3-bf19-c563e2e95d07 admin admin] PATH_INFO: `/v3/projects/7394ae5eaff64d52a5d4f49fb4cb58d1` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:54.688305 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fdcfd3-cecc-46d3-bf19-c563e2e95d07 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7394ae5eaff64d52a5d4f49fb4cb58d1)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:54.690360 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fdcfd3-cecc-46d3-bf19-c563e2e95d07 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:54.944383 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-83fdcfd3-cecc-46d3-bf19-c563e2e95d07 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7394ae5eaff64d52a5d4f49fb4cb58d1', 'request_id': 'req-83fdcfd3-cecc-46d3-bf19-c563e2e95d07'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:36:55.047151 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-83fdcfd3-cecc-46d3-bf19-c563e2e95d07 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7394ae5eaff64d52a5d4f49fb4cb58d1', 'request_id': 'req-83fdcfd3-cecc-46d3-bf19-c563e2e95d07'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:36:55.072164 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-83fdcfd3-cecc-46d3-bf19-c563e2e95d07 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7394ae5eaff64d52a5d4f49fb4cb58d1', 'request_id': 'req-83fdcfd3-cecc-46d3-bf19-c563e2e95d07'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:36:55.102524 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-83fdcfd3-cecc-46d3-bf19-c563e2e95d07 admin admin] The token cache is being invalidate because project 7394ae5eaff64d52a5d4f49fb4cb58d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81731) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:36:55.109736 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-83fdcfd3-cecc-46d3-bf19-c563e2e95d07 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:36:55.109736 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-83fdcfd3-cecc-46d3-bf19-c563e2e95d07 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:36:55.109736 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1200/2374] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 24 17:36:54 2022] DELETE /identity/v3/projects/7394ae5eaff64d52a5d4f49fb4cb58d1 => generated 0 bytes in 562 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:36:55.112445 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127cb77c-6270-4526-8d2b-06b3807aa3a1 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:55.113360 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127cb77c-6270-4526-8d2b-06b3807aa3a1 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:55.113982 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127cb77c-6270-4526-8d2b-06b3807aa3a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:55.147520 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-127cb77c-6270-4526-8d2b-06b3807aa3a1 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:36:55.149741 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-127cb77c-6270-4526-8d2b-06b3807aa3a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:36:55.278799 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1175/2375] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:36:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:36:55.287037 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94906030-5047-46c6-ad7a-311994532b19 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:55.287818 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94906030-5047-46c6-ad7a-311994532b19 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:55.288306 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94906030-5047-46c6-ad7a-311994532b19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:55.317324 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-94906030-5047-46c6-ad7a-311994532b19 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:36:55.318579 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-94906030-5047-46c6-ad7a-311994532b19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:36:55.339716 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1201/2376] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:36:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:36:55.346484 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127cb77c-6270-4526-8d2b-06b3807aa3a1 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:55.362520 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127cb77c-6270-4526-8d2b-06b3807aa3a1 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:55.370363 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02842c7d-81c3-41f2-a322-0e31e55d8e54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:55.371041 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02842c7d-81c3-41f2-a322-0e31e55d8e54 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:55.371345 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02842c7d-81c3-41f2-a322-0e31e55d8e54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:55.371606 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02842c7d-81c3-41f2-a322-0e31e55d8e54 admin admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:55.373714 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02842c7d-81c3-41f2-a322-0e31e55d8e54 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:36:55.373886 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02842c7d-81c3-41f2-a322-0e31e55d8e54 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:55.382958 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02842c7d-81c3-41f2-a322-0e31e55d8e54 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:55.407867 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1176/2377] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 24 17:36:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:36:55.412446 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94906030-5047-46c6-ad7a-311994532b19 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:55.427217 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94906030-5047-46c6-ad7a-311994532b19 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:55.445026 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e37aedf-489c-4a1d-91c6-fa8cd88429f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:55.445839 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e37aedf-489c-4a1d-91c6-fa8cd88429f8 admin admin] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:55.446148 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e37aedf-489c-4a1d-91c6-fa8cd88429f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:55.446423 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e37aedf-489c-4a1d-91c6-fa8cd88429f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:55.447261 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e37aedf-489c-4a1d-91c6-fa8cd88429f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:55.451036 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e37aedf-489c-4a1d-91c6-fa8cd88429f8 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:55.590061 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1202/2378] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 24 17:36:55 2022] POST /identity/v3/projects => generated 363 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:36:55.594871 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02842c7d-81c3-41f2-a322-0e31e55d8e54 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:55.612086 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02842c7d-81c3-41f2-a322-0e31e55d8e54 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:55.696038 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:55.722646 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f62448f-4c8d-4293-b7b8-1fabddba02c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:55.723301 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f62448f-4c8d-4293-b7b8-1fabddba02c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:55.723576 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f62448f-4c8d-4293-b7b8-1fabddba02c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:55.723850 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f62448f-4c8d-4293-b7b8-1fabddba02c5 admin admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:55.724470 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f62448f-4c8d-4293-b7b8-1fabddba02c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:55.726499 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f62448f-4c8d-4293-b7b8-1fabddba02c5 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:56.133750 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1177/2379] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 24 17:36:55 2022] POST /identity/v3/users => generated 282 bytes in 539 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:36:56.138013 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e37aedf-489c-4a1d-91c6-fa8cd88429f8 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:56.157466 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e37aedf-489c-4a1d-91c6-fa8cd88429f8 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:56.162709 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e11098-69a9-4747-952a-01ec2984b752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:56.163347 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e11098-69a9-4747-952a-01ec2984b752 admin admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:56.163673 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e11098-69a9-4747-952a-01ec2984b752 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:56.163948 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e11098-69a9-4747-952a-01ec2984b752 admin admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:56.164609 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e11098-69a9-4747-952a-01ec2984b752 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:56.167314 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e11098-69a9-4747-952a-01ec2984b752 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:56.177969 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1203/2380] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:36:56 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:36:56.184079 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f62448f-4c8d-4293-b7b8-1fabddba02c5 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:56.197527 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f62448f-4c8d-4293-b7b8-1fabddba02c5 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:56.202807 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0112a4-5ed1-4c17-a5f9-3c5b98848980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:56.203434 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0112a4-5ed1-4c17-a5f9-3c5b98848980 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:56.203711 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0112a4-5ed1-4c17-a5f9-3c5b98848980 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:56.203981 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0112a4-5ed1-4c17-a5f9-3c5b98848980 admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:56.204639 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0112a4-5ed1-4c17-a5f9-3c5b98848980 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:56.206293 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0112a4-5ed1-4c17-a5f9-3c5b98848980 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:56.217139 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1178/2381] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:36:56 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:36:56.221437 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e11098-69a9-4747-952a-01ec2984b752 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:56.232160 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e11098-69a9-4747-952a-01ec2984b752 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:56.250042 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b03b70-0488-4292-bb0e-a3bd40258913 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:56.250042 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b03b70-0488-4292-bb0e-a3bd40258913 admin admin] REQUEST_METHOD: `PUT` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:56.250042 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b03b70-0488-4292-bb0e-a3bd40258913 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:56.250042 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b03b70-0488-4292-bb0e-a3bd40258913 admin admin] PATH_INFO: `/v3/projects/59647c15346e4483b3154df46fd27d37/users/5fbdd4c8b5da41db88cf3d3a33eb1e8d/roles/60dc7e16aee746d5a3e8aa27d7d8525c` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:56.264112 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b03b70-0488-4292-bb0e-a3bd40258913 admin admin] RBAC: Authorizing `identity:create_grant(project_id=59647c15346e4483b3154df46fd27d37, user_id=5fbdd4c8b5da41db88cf3d3a33eb1e8d, role_id=60dc7e16aee746d5a3e8aa27d7d8525c)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:56.266738 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b03b70-0488-4292-bb0e-a3bd40258913 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:56.387894 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1204/2382] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 24 17:36:56 2022] PUT /identity/v3/projects/59647c15346e4483b3154df46fd27d37/users/5fbdd4c8b5da41db88cf3d3a33eb1e8d/roles/60dc7e16aee746d5a3e8aa27d7d8525c => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:36:56.395517 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb78944-c1fb-4d0a-a587-c5fee0adf69a None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:56.395787 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb78944-c1fb-4d0a-a587-c5fee0adf69a None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:56.396036 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb78944-c1fb-4d0a-a587-c5fee0adf69a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:56.445015 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-ffb78944-c1fb-4d0a-a587-c5fee0adf69a None None] MFA Rules not processed for user `5fbdd4c8b5da41db88cf3d3a33eb1e8d`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:36:56.446334 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-ffb78944-c1fb-4d0a-a587-c5fee0adf69a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:36:56.600114 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1179/2383] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:36:56 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:36:56.604302 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b03b70-0488-4292-bb0e-a3bd40258913 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:56.630195 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=88,keystone:DELETE=9,keystone:INSERT=3 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:56.761014 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b03b70-0488-4292-bb0e-a3bd40258913 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:56.770993 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e551196d-8f2b-48b1-9628-15fb442b1939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:56.771780 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e551196d-8f2b-48b1-9628-15fb442b1939 admin admin] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:56.772021 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e551196d-8f2b-48b1-9628-15fb442b1939 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:56.772276 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e551196d-8f2b-48b1-9628-15fb442b1939 admin admin] PATH_INFO: `/v3/projects` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:56.772993 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e551196d-8f2b-48b1-9628-15fb442b1939 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:56.775138 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e551196d-8f2b-48b1-9628-15fb442b1939 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:56.888416 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1205/2384] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 24 17:36:56 2022] POST /identity/v3/projects => generated 363 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:36:56.893481 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb78944-c1fb-4d0a-a587-c5fee0adf69a None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:56.935684 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb78944-c1fb-4d0a-a587-c5fee0adf69a None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:57.068266 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c575731-09cb-4732-a26c-dba45ce8e381 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:57.072048 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c575731-09cb-4732-a26c-dba45ce8e381 admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:57.072048 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c575731-09cb-4732-a26c-dba45ce8e381 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:57.072048 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c575731-09cb-4732-a26c-dba45ce8e381 admin admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:57.074607 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c575731-09cb-4732-a26c-dba45ce8e381 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:57.084299 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c575731-09cb-4732-a26c-dba45ce8e381 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:57.324091 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1180/2385] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 24 17:36:56 2022] POST /identity/v3/users => generated 288 bytes in 432 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:36:57.325252 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e551196d-8f2b-48b1-9628-15fb442b1939 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:57.341518 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e551196d-8f2b-48b1-9628-15fb442b1939 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:57.359150 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e580f4-57cd-4dc5-94c2-7bc13d7246ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:57.360098 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e580f4-57cd-4dc5-94c2-7bc13d7246ea admin admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:57.360602 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e580f4-57cd-4dc5-94c2-7bc13d7246ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:57.361113 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e580f4-57cd-4dc5-94c2-7bc13d7246ea admin admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:57.362833 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e580f4-57cd-4dc5-94c2-7bc13d7246ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:57.366047 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e580f4-57cd-4dc5-94c2-7bc13d7246ea admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:57.391870 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1206/2386] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:36:57 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:36:57.394024 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c575731-09cb-4732-a26c-dba45ce8e381 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:57.414874 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c575731-09cb-4732-a26c-dba45ce8e381 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:57.422309 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e444c99f-6738-4ccb-b73e-ba8447db2472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:57.423499 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e444c99f-6738-4ccb-b73e-ba8447db2472 admin admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:57.424023 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e444c99f-6738-4ccb-b73e-ba8447db2472 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:57.424542 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e444c99f-6738-4ccb-b73e-ba8447db2472 admin admin] PATH_INFO: `/v3/projects/993e7e4aeea849688394a24f27412793/users/72b79d0071a3418199b909f2803dd3c9/roles/c5346bc54abf445f9900edf94264c850` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:57.455042 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e444c99f-6738-4ccb-b73e-ba8447db2472 admin admin] RBAC: Authorizing `identity:create_grant(project_id=993e7e4aeea849688394a24f27412793, user_id=72b79d0071a3418199b909f2803dd3c9, role_id=c5346bc54abf445f9900edf94264c850)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:57.459886 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e444c99f-6738-4ccb-b73e-ba8447db2472 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:57.585946 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1181/2387] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 24 17:36:57 2022] PUT /identity/v3/projects/993e7e4aeea849688394a24f27412793/users/72b79d0071a3418199b909f2803dd3c9/roles/c5346bc54abf445f9900edf94264c850 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:36:57.590353 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e580f4-57cd-4dc5-94c2-7bc13d7246ea admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:57.610674 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e580f4-57cd-4dc5-94c2-7bc13d7246ea admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:57.719827 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ced48e-c615-4ac8-938d-294e3e3ef2d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:57.721420 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ced48e-c615-4ac8-938d-294e3e3ef2d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:57.723928 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ced48e-c615-4ac8-938d-294e3e3ef2d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:57.724864 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ced48e-c615-4ac8-938d-294e3e3ef2d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:57.726021 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ced48e-c615-4ac8-938d-294e3e3ef2d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:57.728835 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ced48e-c615-4ac8-938d-294e3e3ef2d4 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:57.745267 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1207/2388] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:36:57 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:36:57.751736 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e444c99f-6738-4ccb-b73e-ba8447db2472 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:57.771080 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e444c99f-6738-4ccb-b73e-ba8447db2472 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:57.782235 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2850c2d-7cf0-4aaa-91c7-c3e7b3b1c8e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:57.783814 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2850c2d-7cf0-4aaa-91c7-c3e7b3b1c8e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:57.784732 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2850c2d-7cf0-4aaa-91c7-c3e7b3b1c8e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:57.785206 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2850c2d-7cf0-4aaa-91c7-c3e7b3b1c8e4 admin admin] PATH_INFO: `/v3/projects/993e7e4aeea849688394a24f27412793/users/72b79d0071a3418199b909f2803dd3c9/roles/c5346bc54abf445f9900edf94264c850` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:57.788335 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2850c2d-7cf0-4aaa-91c7-c3e7b3b1c8e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=993e7e4aeea849688394a24f27412793, user_id=72b79d0071a3418199b909f2803dd3c9, role_id=c5346bc54abf445f9900edf94264c850)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:57.793091 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2850c2d-7cf0-4aaa-91c7-c3e7b3b1c8e4 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:57.805064 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:57.811828 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1182/2389] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 24 17:36:57 2022] PUT /identity/v3/projects/993e7e4aeea849688394a24f27412793/users/72b79d0071a3418199b909f2803dd3c9/roles/c5346bc54abf445f9900edf94264c850 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:36:57.825446 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97e24a5-a107-4435-9792-1f2003231768 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:57.831896 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97e24a5-a107-4435-9792-1f2003231768 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:57.831896 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97e24a5-a107-4435-9792-1f2003231768 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:57.879929 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-f97e24a5-a107-4435-9792-1f2003231768 None None] MFA Rules not processed for user `72b79d0071a3418199b909f2803dd3c9`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:36:57.885343 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-f97e24a5-a107-4435-9792-1f2003231768 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:36:58.052188 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1208/2390] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:36:57 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:36:58.064732 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2850c2d-7cf0-4aaa-91c7-c3e7b3b1c8e4 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:58.068336 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-f2850c2d-7cf0-4aaa-91c7-c3e7b3b1c8e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:36:58.254142 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2850c2d-7cf0-4aaa-91c7-c3e7b3b1c8e4 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:58.284317 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9b7396-ac88-4538-a8a7-ab44d2b33990 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be06246d47584b529150ceb98a5cfd95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:58.284317 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9b7396-ac88-4538-a8a7-ab44d2b33990 service cinder] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:58.284317 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9b7396-ac88-4538-a8a7-ab44d2b33990 service cinder] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:58.284317 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9b7396-ac88-4538-a8a7-ab44d2b33990 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:58.330369 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9b7396-ac88-4538-a8a7-ab44d2b33990 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:58.336426 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9b7396-ac88-4538-a8a7-ab44d2b33990 service cinder] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:58.371330 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1183/2391] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:36:58 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:36:58.586121 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97e24a5-a107-4435-9792-1f2003231768 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:36:58.595143 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:36:58.606271 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97e24a5-a107-4435-9792-1f2003231768 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:36:58.612777 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4784d0a-d42f-4620-bcdc-c356db3751ce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be06246d47584b529150ceb98a5cfd95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:36:58.612777 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4784d0a-d42f-4620-bcdc-c356db3751ce service cinder] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:36:58.612777 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4784d0a-d42f-4620-bcdc-c356db3751ce service cinder] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:36:58.612777 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4784d0a-d42f-4620-bcdc-c356db3751ce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:36:58.729254 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4784d0a-d42f-4620-bcdc-c356db3751ce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:36:58.731895 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4784d0a-d42f-4620-bcdc-c356db3751ce service cinder] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:36:58.748750 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1209/2392] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:36:58 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:37:03.098830 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9b7396-ac88-4538-a8a7-ab44d2b33990 service cinder] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:37:03.228221 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9b7396-ac88-4538-a8a7-ab44d2b33990 service cinder] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:37:03.236790 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df19093-5205-45b5-a475-96f487d81b01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:37:03.237904 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df19093-5205-45b5-a475-96f487d81b01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:03.238496 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df19093-5205-45b5-a475-96f487d81b01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:03.239036 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df19093-5205-45b5-a475-96f487d81b01 admin admin] PATH_INFO: `/v3/users/5fbdd4c8b5da41db88cf3d3a33eb1e8d` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:03.240741 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df19093-5205-45b5-a475-96f487d81b01 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5fbdd4c8b5da41db88cf3d3a33eb1e8d)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:37:03.243218 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df19093-5205-45b5-a475-96f487d81b01 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:37:03.488660 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-9df19093-5205-45b5-a475-96f487d81b01 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5fbdd4c8b5da41db88cf3d3a33eb1e8d'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:37:03.563306 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-9df19093-5205-45b5-a475-96f487d81b01 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5fbdd4c8b5da41db88cf3d3a33eb1e8d'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:37:03.565064 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-9df19093-5205-45b5-a475-96f487d81b01 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5fbdd4c8b5da41db88cf3d3a33eb1e8d'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:37:03.584353 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-9df19093-5205-45b5-a475-96f487d81b01 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5fbdd4c8b5da41db88cf3d3a33eb1e8d'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:37:03.585206 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-9df19093-5205-45b5-a475-96f487d81b01 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5fbdd4c8b5da41db88cf3d3a33eb1e8d'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:37:03.622281 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1184/2393] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 24 17:37:03 2022] DELETE /identity/v3/users/5fbdd4c8b5da41db88cf3d3a33eb1e8d => generated 0 bytes in 525 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:37:03.628860 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062fa5a4-84c7-4bbd-912b-a39d6ce61a99 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:03.629400 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062fa5a4-84c7-4bbd-912b-a39d6ce61a99 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:03.629831 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062fa5a4-84c7-4bbd-912b-a39d6ce61a99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:03.667683 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-062fa5a4-84c7-4bbd-912b-a39d6ce61a99 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:37:03.669792 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-062fa5a4-84c7-4bbd-912b-a39d6ce61a99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:37:03.797697 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1210/2394] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:37:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:37:03.809593 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df19093-5205-45b5-a475-96f487d81b01 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:37:03.814570 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-9df19093-5205-45b5-a475-96f487d81b01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:37:03.845939 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:37:03.908948 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df19093-5205-45b5-a475-96f487d81b01 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:37:03.916613 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec529b44-7c04-4497-8cbf-1b69e74e03e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e9ed4d47aef4140bfb2d3bb798cc559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:37:03.917238 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec529b44-7c04-4497-8cbf-1b69e74e03e5 service neutron] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:03.917510 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec529b44-7c04-4497-8cbf-1b69e74e03e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:03.917780 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec529b44-7c04-4497-8cbf-1b69e74e03e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:03.938937 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec529b44-7c04-4497-8cbf-1b69e74e03e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:37:03.943491 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec529b44-7c04-4497-8cbf-1b69e74e03e5 service neutron] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:37:03.958896 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1185/2395] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:37:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:37:04.668927 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062fa5a4-84c7-4bbd-912b-a39d6ce61a99 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:37:04.671909 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-062fa5a4-84c7-4bbd-912b-a39d6ce61a99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:37:04.685853 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062fa5a4-84c7-4bbd-912b-a39d6ce61a99 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:37:04.691325 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577d2df8-2e18-44ed-b947-0c7d9af0ee4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:37:04.692042 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577d2df8-2e18-44ed-b947-0c7d9af0ee4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:04.692320 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577d2df8-2e18-44ed-b947-0c7d9af0ee4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:04.692589 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577d2df8-2e18-44ed-b947-0c7d9af0ee4c admin admin] PATH_INFO: `/v3/projects/59647c15346e4483b3154df46fd27d37` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:04.693764 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577d2df8-2e18-44ed-b947-0c7d9af0ee4c admin admin] RBAC: Authorizing `identity:delete_project(project_id=59647c15346e4483b3154df46fd27d37)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:37:04.695807 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577d2df8-2e18-44ed-b947-0c7d9af0ee4c admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:37:04.881842 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-577d2df8-2e18-44ed-b947-0c7d9af0ee4c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '59647c15346e4483b3154df46fd27d37', 'request_id': 'req-577d2df8-2e18-44ed-b947-0c7d9af0ee4c'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:37:04.943829 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-577d2df8-2e18-44ed-b947-0c7d9af0ee4c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '59647c15346e4483b3154df46fd27d37', 'request_id': 'req-577d2df8-2e18-44ed-b947-0c7d9af0ee4c'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:37:04.952562 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-577d2df8-2e18-44ed-b947-0c7d9af0ee4c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '59647c15346e4483b3154df46fd27d37', 'request_id': 'req-577d2df8-2e18-44ed-b947-0c7d9af0ee4c'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:37:04.969904 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-577d2df8-2e18-44ed-b947-0c7d9af0ee4c admin admin] The token cache is being invalidate because project 59647c15346e4483b3154df46fd27d37 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81731) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:37:04.970359 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-577d2df8-2e18-44ed-b947-0c7d9af0ee4c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:37:04.972677 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-577d2df8-2e18-44ed-b947-0c7d9af0ee4c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:37:04.975293 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1211/2396] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 24 17:37:04 2022] DELETE /identity/v3/projects/59647c15346e4483b3154df46fd27d37 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:37:04.979575 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec529b44-7c04-4497-8cbf-1b69e74e03e5 service neutron] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:37:04.992013 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec529b44-7c04-4497-8cbf-1b69e74e03e5 service neutron] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:37:05.094347 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f70899b-6607-4d5a-adfc-c09d771705da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:37:05.095358 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f70899b-6607-4d5a-adfc-c09d771705da admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:05.095787 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f70899b-6607-4d5a-adfc-c09d771705da admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:05.096131 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f70899b-6607-4d5a-adfc-c09d771705da admin admin] PATH_INFO: `/v3/users/72b79d0071a3418199b909f2803dd3c9` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:05.098746 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f70899b-6607-4d5a-adfc-c09d771705da admin admin] RBAC: Authorizing `identity:delete_user(user_id=72b79d0071a3418199b909f2803dd3c9)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:37:05.101893 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f70899b-6607-4d5a-adfc-c09d771705da admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:37:05.372039 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-4f70899b-6607-4d5a-adfc-c09d771705da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72b79d0071a3418199b909f2803dd3c9'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:37:05.443081 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-4f70899b-6607-4d5a-adfc-c09d771705da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72b79d0071a3418199b909f2803dd3c9'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:37:05.444103 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-4f70899b-6607-4d5a-adfc-c09d771705da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72b79d0071a3418199b909f2803dd3c9'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:37:05.473723 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-4f70899b-6607-4d5a-adfc-c09d771705da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72b79d0071a3418199b909f2803dd3c9'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:37:05.475207 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-4f70899b-6607-4d5a-adfc-c09d771705da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72b79d0071a3418199b909f2803dd3c9'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:37:05.522351 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1186/2397] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 24 17:37:04 2022] DELETE /identity/v3/users/72b79d0071a3418199b909f2803dd3c9 => generated 0 bytes in 543 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:37:06.257345 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577d2df8-2e18-44ed-b947-0c7d9af0ee4c admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:37:06.259809 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-577d2df8-2e18-44ed-b947-0c7d9af0ee4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:37:06.305299 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577d2df8-2e18-44ed-b947-0c7d9af0ee4c admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:37:06.359017 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:37:06.417235 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cf11fb-4f7d-4f86-b7fd-fa0dee535b25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:37:06.417898 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cf11fb-4f7d-4f86-b7fd-fa0dee535b25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:06.418174 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cf11fb-4f7d-4f86-b7fd-fa0dee535b25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:06.418443 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cf11fb-4f7d-4f86-b7fd-fa0dee535b25 admin admin] PATH_INFO: `/v3/projects/993e7e4aeea849688394a24f27412793` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:06.419865 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cf11fb-4f7d-4f86-b7fd-fa0dee535b25 admin admin] RBAC: Authorizing `identity:delete_project(project_id=993e7e4aeea849688394a24f27412793)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:37:06.421838 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cf11fb-4f7d-4f86-b7fd-fa0dee535b25 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:37:06.547689 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-33cf11fb-4f7d-4f86-b7fd-fa0dee535b25 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '993e7e4aeea849688394a24f27412793', 'request_id': 'req-33cf11fb-4f7d-4f86-b7fd-fa0dee535b25'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:37:06.635338 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-33cf11fb-4f7d-4f86-b7fd-fa0dee535b25 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '993e7e4aeea849688394a24f27412793', 'request_id': 'req-33cf11fb-4f7d-4f86-b7fd-fa0dee535b25'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:37:06.646840 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-33cf11fb-4f7d-4f86-b7fd-fa0dee535b25 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '993e7e4aeea849688394a24f27412793', 'request_id': 'req-33cf11fb-4f7d-4f86-b7fd-fa0dee535b25'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:37:06.665029 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-33cf11fb-4f7d-4f86-b7fd-fa0dee535b25 admin admin] The token cache is being invalidate because project 993e7e4aeea849688394a24f27412793 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81731) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:37:06.665506 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-33cf11fb-4f7d-4f86-b7fd-fa0dee535b25 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:37:06.666763 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-33cf11fb-4f7d-4f86-b7fd-fa0dee535b25 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:37:06.669143 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1212/2398] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 24 17:37:06 2022] DELETE /identity/v3/projects/993e7e4aeea849688394a24f27412793 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:37:06.676734 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f694566-a0fe-4a98-a9ca-fa74ebdde501 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:06.677516 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f694566-a0fe-4a98-a9ca-fa74ebdde501 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:06.677704 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f694566-a0fe-4a98-a9ca-fa74ebdde501 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:06.710964 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-6f694566-a0fe-4a98-a9ca-fa74ebdde501 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:37:06.712197 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-6f694566-a0fe-4a98-a9ca-fa74ebdde501 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:37:06.758780 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:37:06.789618 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1187/2399] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:37:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:37:06.798127 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626e297e-235f-4b9b-a607-1985717a817b None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:06.798398 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626e297e-235f-4b9b-a607-1985717a817b None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:06.798705 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626e297e-235f-4b9b-a607-1985717a817b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:06.829061 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-626e297e-235f-4b9b-a607-1985717a817b None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:37:06.831525 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-626e297e-235f-4b9b-a607-1985717a817b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:37:06.854812 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1213/2400] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:37:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:37:06.855911 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f694566-a0fe-4a98-a9ca-fa74ebdde501 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:37:06.868091 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f694566-a0fe-4a98-a9ca-fa74ebdde501 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:37:06.875188 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2f5097-4b64-4719-80ca-2d9ff84458f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:37:06.875968 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2f5097-4b64-4719-80ca-2d9ff84458f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:06.876457 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2f5097-4b64-4719-80ca-2d9ff84458f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:06.876883 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2f5097-4b64-4719-80ca-2d9ff84458f0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:06.878473 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2f5097-4b64-4719-80ca-2d9ff84458f0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:37:06.879007 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2f5097-4b64-4719-80ca-2d9ff84458f0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:37:06.881352 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2f5097-4b64-4719-80ca-2d9ff84458f0 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:37:06.901865 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1188/2401] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 24 17:37:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:37:06.906577 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626e297e-235f-4b9b-a607-1985717a817b None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:37:06.922618 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626e297e-235f-4b9b-a607-1985717a817b None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:37:06.930806 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44c866e-9152-40c6-b5c6-bd2a98ce9e78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:37:06.931756 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44c866e-9152-40c6-b5c6-bd2a98ce9e78 admin admin] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:06.932200 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44c866e-9152-40c6-b5c6-bd2a98ce9e78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:06.932677 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44c866e-9152-40c6-b5c6-bd2a98ce9e78 admin admin] PATH_INFO: `/v3/projects` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:06.933819 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44c866e-9152-40c6-b5c6-bd2a98ce9e78 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:37:06.937509 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44c866e-9152-40c6-b5c6-bd2a98ce9e78 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:37:07.085619 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1214/2402] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 24 17:37:06 2022] POST /identity/v3/projects => generated 387 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:37:07.087380 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2f5097-4b64-4719-80ca-2d9ff84458f0 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:37:07.106121 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2f5097-4b64-4719-80ca-2d9ff84458f0 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:37:07.241291 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa0bc45-2527-4f53-ac03-1fa35f3e93c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:37:07.242270 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa0bc45-2527-4f53-ac03-1fa35f3e93c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:07.242737 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa0bc45-2527-4f53-ac03-1fa35f3e93c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:07.243217 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa0bc45-2527-4f53-ac03-1fa35f3e93c0 admin admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:07.244244 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa0bc45-2527-4f53-ac03-1fa35f3e93c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:37:07.248750 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa0bc45-2527-4f53-ac03-1fa35f3e93c0 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:37:07.336261 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1189/2403] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 24 17:37:07 2022] POST /identity/v3/users => generated 294 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:37:07.338517 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44c866e-9152-40c6-b5c6-bd2a98ce9e78 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:37:07.375600 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44c866e-9152-40c6-b5c6-bd2a98ce9e78 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:37:07.388295 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2f08b7-0207-40c6-81ac-c9c45cba0a0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:37:07.390263 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2f08b7-0207-40c6-81ac-c9c45cba0a0e admin admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:07.393842 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2f08b7-0207-40c6-81ac-c9c45cba0a0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:07.394611 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2f08b7-0207-40c6-81ac-c9c45cba0a0e admin admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:07.395863 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2f08b7-0207-40c6-81ac-c9c45cba0a0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:37:07.400470 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2f08b7-0207-40c6-81ac-c9c45cba0a0e admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:37:07.441042 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1215/2404] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:37:07 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:37:07.445447 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa0bc45-2527-4f53-ac03-1fa35f3e93c0 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:37:07.457243 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa0bc45-2527-4f53-ac03-1fa35f3e93c0 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:37:07.464832 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8654c37c-d2bb-42c5-be1b-a3d87cf41b4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:37:07.465184 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8654c37c-d2bb-42c5-be1b-a3d87cf41b4b admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:07.465580 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8654c37c-d2bb-42c5-be1b-a3d87cf41b4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:07.465963 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8654c37c-d2bb-42c5-be1b-a3d87cf41b4b admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:07.466777 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8654c37c-d2bb-42c5-be1b-a3d87cf41b4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:37:07.468675 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8654c37c-d2bb-42c5-be1b-a3d87cf41b4b admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:37:07.488284 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1190/2405] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:37:07 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:37:07.494090 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2f08b7-0207-40c6-81ac-c9c45cba0a0e admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:37:07.505209 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2f08b7-0207-40c6-81ac-c9c45cba0a0e admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:37:07.512496 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbac4f7f-1cfc-4e1f-82e4-16a8d5ac1eff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:37:07.513569 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbac4f7f-1cfc-4e1f-82e4-16a8d5ac1eff admin admin] REQUEST_METHOD: `PUT` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:07.514014 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbac4f7f-1cfc-4e1f-82e4-16a8d5ac1eff admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:07.514456 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbac4f7f-1cfc-4e1f-82e4-16a8d5ac1eff admin admin] PATH_INFO: `/v3/projects/a8605bf7e745401d81289a9f30a0ab0e/users/d896181bb65a4cea9f65273b5adafb20/roles/60dc7e16aee746d5a3e8aa27d7d8525c` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:07.541570 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbac4f7f-1cfc-4e1f-82e4-16a8d5ac1eff admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8605bf7e745401d81289a9f30a0ab0e, user_id=d896181bb65a4cea9f65273b5adafb20, role_id=60dc7e16aee746d5a3e8aa27d7d8525c)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:37:07.547831 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbac4f7f-1cfc-4e1f-82e4-16a8d5ac1eff admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:37:07.623221 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1216/2406] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 24 17:37:07 2022] PUT /identity/v3/projects/a8605bf7e745401d81289a9f30a0ab0e/users/d896181bb65a4cea9f65273b5adafb20/roles/60dc7e16aee746d5a3e8aa27d7d8525c => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:37:07.633699 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef7e344-0c36-4249-a1d8-1d1170f693a9 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:07.634020 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef7e344-0c36-4249-a1d8-1d1170f693a9 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:07.634254 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef7e344-0c36-4249-a1d8-1d1170f693a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:07.671387 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-6ef7e344-0c36-4249-a1d8-1d1170f693a9 None None] MFA Rules not processed for user `d896181bb65a4cea9f65273b5adafb20`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:37:07.672534 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-6ef7e344-0c36-4249-a1d8-1d1170f693a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:37:07.766495 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1191/2407] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:37:07 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:37:07.773545 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbac4f7f-1cfc-4e1f-82e4-16a8d5ac1eff admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:37:07.872415 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:37:07.890175 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbac4f7f-1cfc-4e1f-82e4-16a8d5ac1eff admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:37:07.896664 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be1adfd-7be7-459a-b5a0-b009684005bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:37:07.897398 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be1adfd-7be7-459a-b5a0-b009684005bd admin admin] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:07.897776 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be1adfd-7be7-459a-b5a0-b009684005bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:07.898095 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be1adfd-7be7-459a-b5a0-b009684005bd admin admin] PATH_INFO: `/v3/projects` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:07.898728 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be1adfd-7be7-459a-b5a0-b009684005bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:37:07.900671 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be1adfd-7be7-459a-b5a0-b009684005bd admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:37:07.998340 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1217/2408] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 24 17:37:07 2022] POST /identity/v3/projects => generated 387 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:37:08.003132 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef7e344-0c36-4249-a1d8-1d1170f693a9 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:37:08.016192 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef7e344-0c36-4249-a1d8-1d1170f693a9 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:37:08.069933 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:37:08.111115 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b632b9-9ada-4b63-b20b-207b5748c89e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:37:08.111796 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b632b9-9ada-4b63-b20b-207b5748c89e admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:08.112086 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b632b9-9ada-4b63-b20b-207b5748c89e admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:08.112453 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b632b9-9ada-4b63-b20b-207b5748c89e admin admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:08.113231 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b632b9-9ada-4b63-b20b-207b5748c89e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:37:08.115231 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b632b9-9ada-4b63-b20b-207b5748c89e admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:37:08.193057 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1192/2409] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 24 17:37:08 2022] POST /identity/v3/users => generated 300 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:37:08.198084 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be1adfd-7be7-459a-b5a0-b009684005bd admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:37:08.211032 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be1adfd-7be7-459a-b5a0-b009684005bd admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:37:08.217011 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ac0c90-4ec2-4970-9dc9-3aba18cea009 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:37:08.218051 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ac0c90-4ec2-4970-9dc9-3aba18cea009 admin admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:08.218468 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ac0c90-4ec2-4970-9dc9-3aba18cea009 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:08.218874 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ac0c90-4ec2-4970-9dc9-3aba18cea009 admin admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:08.219839 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ac0c90-4ec2-4970-9dc9-3aba18cea009 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:37:08.222880 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ac0c90-4ec2-4970-9dc9-3aba18cea009 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:37:08.244059 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1218/2410] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:37:08 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:37:08.250134 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b632b9-9ada-4b63-b20b-207b5748c89e admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:37:08.264784 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b632b9-9ada-4b63-b20b-207b5748c89e admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:37:08.270590 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2891ca25-e4f9-4b8f-a60f-7e44023b202f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:37:08.271320 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2891ca25-e4f9-4b8f-a60f-7e44023b202f admin admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:08.271612 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2891ca25-e4f9-4b8f-a60f-7e44023b202f admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:08.271893 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2891ca25-e4f9-4b8f-a60f-7e44023b202f admin admin] PATH_INFO: `/v3/projects/739bb819cf204184b577c0fcd9bd5ec1/users/d250c259c0e5488bb74fbcbc858e087a/roles/c5346bc54abf445f9900edf94264c850` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:08.290833 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2891ca25-e4f9-4b8f-a60f-7e44023b202f admin admin] RBAC: Authorizing `identity:create_grant(project_id=739bb819cf204184b577c0fcd9bd5ec1, user_id=d250c259c0e5488bb74fbcbc858e087a, role_id=c5346bc54abf445f9900edf94264c850)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:37:08.293601 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2891ca25-e4f9-4b8f-a60f-7e44023b202f admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:37:08.369903 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1193/2411] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 24 17:37:08 2022] PUT /identity/v3/projects/739bb819cf204184b577c0fcd9bd5ec1/users/d250c259c0e5488bb74fbcbc858e087a/roles/c5346bc54abf445f9900edf94264c850 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:37:08.375800 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ac0c90-4ec2-4970-9dc9-3aba18cea009 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:37:08.389624 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ac0c90-4ec2-4970-9dc9-3aba18cea009 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:37:08.469107 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15630e5-2f2b-4d54-94fb-0e708f48f604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:37:08.469660 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15630e5-2f2b-4d54-94fb-0e708f48f604 admin admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:08.470097 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15630e5-2f2b-4d54-94fb-0e708f48f604 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:08.470411 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15630e5-2f2b-4d54-94fb-0e708f48f604 admin admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:08.471621 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15630e5-2f2b-4d54-94fb-0e708f48f604 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:37:08.474597 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15630e5-2f2b-4d54-94fb-0e708f48f604 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:37:08.499175 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1219/2412] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:37:08 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:37:08.502095 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2891ca25-e4f9-4b8f-a60f-7e44023b202f admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:37:08.518271 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2891ca25-e4f9-4b8f-a60f-7e44023b202f admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:37:08.525297 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b003e7-6dcb-47a9-b706-db109b50365f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:37:08.526884 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b003e7-6dcb-47a9-b706-db109b50365f admin admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:08.527700 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b003e7-6dcb-47a9-b706-db109b50365f admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:08.528132 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b003e7-6dcb-47a9-b706-db109b50365f admin admin] PATH_INFO: `/v3/projects/739bb819cf204184b577c0fcd9bd5ec1/users/d250c259c0e5488bb74fbcbc858e087a/roles/c5346bc54abf445f9900edf94264c850` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:08.531407 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b003e7-6dcb-47a9-b706-db109b50365f admin admin] RBAC: Authorizing `identity:create_grant(project_id=739bb819cf204184b577c0fcd9bd5ec1, user_id=d250c259c0e5488bb74fbcbc858e087a, role_id=c5346bc54abf445f9900edf94264c850)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:37:08.536893 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b003e7-6dcb-47a9-b706-db109b50365f admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:37:08.554693 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1194/2413] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 24 17:37:08 2022] PUT /identity/v3/projects/739bb819cf204184b577c0fcd9bd5ec1/users/d250c259c0e5488bb74fbcbc858e087a/roles/c5346bc54abf445f9900edf94264c850 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:37:08.643477 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2418efa3-af6d-4af2-90bd-8b05263ef6f7 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:08.644200 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2418efa3-af6d-4af2-90bd-8b05263ef6f7 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:08.644689 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2418efa3-af6d-4af2-90bd-8b05263ef6f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:08.705308 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-2418efa3-af6d-4af2-90bd-8b05263ef6f7 None None] MFA Rules not processed for user `d250c259c0e5488bb74fbcbc858e087a`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:37:08.707904 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-2418efa3-af6d-4af2-90bd-8b05263ef6f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:37:08.822940 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1220/2414] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:37:08 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:37:08.833211 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b003e7-6dcb-47a9-b706-db109b50365f admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:37:08.835215 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-00b003e7-6dcb-47a9-b706-db109b50365f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:37:08.911390 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b003e7-6dcb-47a9-b706-db109b50365f admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:37:08.915944 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296226c3-e7d1-451d-81a2-0c068d2b4af0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be06246d47584b529150ceb98a5cfd95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:37:08.916883 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296226c3-e7d1-451d-81a2-0c068d2b4af0 service cinder] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:08.917172 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296226c3-e7d1-451d-81a2-0c068d2b4af0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:08.917451 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296226c3-e7d1-451d-81a2-0c068d2b4af0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:08.930716 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296226c3-e7d1-451d-81a2-0c068d2b4af0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:37:08.932273 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296226c3-e7d1-451d-81a2-0c068d2b4af0 service cinder] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:37:08.943636 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1195/2415] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:37:08 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:37:09.047633 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2418efa3-af6d-4af2-90bd-8b05263ef6f7 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:37:09.140120 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:37:09.166964 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2418efa3-af6d-4af2-90bd-8b05263ef6f7 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:37:09.174335 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c224e3cb-fc14-4445-a6d9-874403cb62ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:37:09.175780 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c224e3cb-fc14-4445-a6d9-874403cb62ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:09.176812 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c224e3cb-fc14-4445-a6d9-874403cb62ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:09.176812 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c224e3cb-fc14-4445-a6d9-874403cb62ef admin admin] PATH_INFO: `/v3/users/d896181bb65a4cea9f65273b5adafb20` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:09.177755 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c224e3cb-fc14-4445-a6d9-874403cb62ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=d896181bb65a4cea9f65273b5adafb20)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:37:09.179815 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c224e3cb-fc14-4445-a6d9-874403cb62ef admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:37:09.391628 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-c224e3cb-fc14-4445-a6d9-874403cb62ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd896181bb65a4cea9f65273b5adafb20'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:37:09.415167 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-c224e3cb-fc14-4445-a6d9-874403cb62ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd896181bb65a4cea9f65273b5adafb20'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:37:09.537877 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-c224e3cb-fc14-4445-a6d9-874403cb62ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd896181bb65a4cea9f65273b5adafb20'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:37:09.538537 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-c224e3cb-fc14-4445-a6d9-874403cb62ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd896181bb65a4cea9f65273b5adafb20'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:37:09.553435 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-c224e3cb-fc14-4445-a6d9-874403cb62ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd896181bb65a4cea9f65273b5adafb20'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:37:09.567952 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1221/2416] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 24 17:37:09 2022] DELETE /identity/v3/users/d896181bb65a4cea9f65273b5adafb20 => generated 0 bytes in 521 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:37:09.572921 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28584cbc-c92d-4a9d-85ea-6103e391389f None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:09.577004 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28584cbc-c92d-4a9d-85ea-6103e391389f None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:09.577004 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28584cbc-c92d-4a9d-85ea-6103e391389f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:09.595661 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-28584cbc-c92d-4a9d-85ea-6103e391389f None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:37:09.597534 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-28584cbc-c92d-4a9d-85ea-6103e391389f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:37:09.675652 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1196/2417] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:37:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:37:09.689232 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c224e3cb-fc14-4445-a6d9-874403cb62ef admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:37:09.691896 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-c224e3cb-fc14-4445-a6d9-874403cb62ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:37:09.771061 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c224e3cb-fc14-4445-a6d9-874403cb62ef admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:37:09.774887 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cce805-1bd3-43ca-bb74-0d098474aaa1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e9ed4d47aef4140bfb2d3bb798cc559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:37:09.775549 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cce805-1bd3-43ca-bb74-0d098474aaa1 service neutron] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:09.776003 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cce805-1bd3-43ca-bb74-0d098474aaa1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:09.776197 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cce805-1bd3-43ca-bb74-0d098474aaa1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:09.786675 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cce805-1bd3-43ca-bb74-0d098474aaa1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:37:09.788171 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cce805-1bd3-43ca-bb74-0d098474aaa1 service neutron] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:37:09.798517 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1222/2418] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:37:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:37:10.326761 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28584cbc-c92d-4a9d-85ea-6103e391389f None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:37:10.328366 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-28584cbc-c92d-4a9d-85ea-6103e391389f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:37:10.343085 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28584cbc-c92d-4a9d-85ea-6103e391389f None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:37:10.348577 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672ba32a-2b99-4ef0-9894-6cdc585c337a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:37:10.349241 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672ba32a-2b99-4ef0-9894-6cdc585c337a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:10.349520 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672ba32a-2b99-4ef0-9894-6cdc585c337a admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:10.349872 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672ba32a-2b99-4ef0-9894-6cdc585c337a admin admin] PATH_INFO: `/v3/projects/a8605bf7e745401d81289a9f30a0ab0e` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:10.351063 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672ba32a-2b99-4ef0-9894-6cdc585c337a admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8605bf7e745401d81289a9f30a0ab0e)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:37:10.353026 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672ba32a-2b99-4ef0-9894-6cdc585c337a admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:37:10.501161 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:37:10.521840 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-672ba32a-2b99-4ef0-9894-6cdc585c337a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8605bf7e745401d81289a9f30a0ab0e', 'request_id': 'req-672ba32a-2b99-4ef0-9894-6cdc585c337a'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:37:10.587568 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-672ba32a-2b99-4ef0-9894-6cdc585c337a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8605bf7e745401d81289a9f30a0ab0e', 'request_id': 'req-672ba32a-2b99-4ef0-9894-6cdc585c337a'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:37:10.594848 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-672ba32a-2b99-4ef0-9894-6cdc585c337a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8605bf7e745401d81289a9f30a0ab0e', 'request_id': 'req-672ba32a-2b99-4ef0-9894-6cdc585c337a'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:37:10.609081 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-672ba32a-2b99-4ef0-9894-6cdc585c337a admin admin] The token cache is being invalidate because project a8605bf7e745401d81289a9f30a0ab0e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81732) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:37:10.609289 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-672ba32a-2b99-4ef0-9894-6cdc585c337a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:37:10.609956 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-672ba32a-2b99-4ef0-9894-6cdc585c337a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:37:10.611529 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1197/2419] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 24 17:37:10 2022] DELETE /identity/v3/projects/a8605bf7e745401d81289a9f30a0ab0e => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:37:10.617180 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cce805-1bd3-43ca-bb74-0d098474aaa1 service neutron] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:37:10.630825 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cce805-1bd3-43ca-bb74-0d098474aaa1 service neutron] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:37:10.701159 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9205f49b-06fb-4e34-849d-12df53e0a55c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:37:10.701818 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9205f49b-06fb-4e34-849d-12df53e0a55c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:10.702093 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9205f49b-06fb-4e34-849d-12df53e0a55c admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:10.702362 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9205f49b-06fb-4e34-849d-12df53e0a55c admin admin] PATH_INFO: `/v3/users/d250c259c0e5488bb74fbcbc858e087a` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:10.703584 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9205f49b-06fb-4e34-849d-12df53e0a55c admin admin] RBAC: Authorizing `identity:delete_user(user_id=d250c259c0e5488bb74fbcbc858e087a)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:37:10.708498 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9205f49b-06fb-4e34-849d-12df53e0a55c admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:37:10.720434 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:37:10.895486 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-9205f49b-06fb-4e34-849d-12df53e0a55c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd250c259c0e5488bb74fbcbc858e087a'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:37:10.915508 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-9205f49b-06fb-4e34-849d-12df53e0a55c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd250c259c0e5488bb74fbcbc858e087a'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:37:10.989278 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-9205f49b-06fb-4e34-849d-12df53e0a55c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd250c259c0e5488bb74fbcbc858e087a'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:37:10.989993 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-9205f49b-06fb-4e34-849d-12df53e0a55c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd250c259c0e5488bb74fbcbc858e087a'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:37:11.004560 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-9205f49b-06fb-4e34-849d-12df53e0a55c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd250c259c0e5488bb74fbcbc858e087a'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:37:11.011005 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1223/2420] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 24 17:37:10 2022] DELETE /identity/v3/users/d250c259c0e5488bb74fbcbc858e087a => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:37:11.562598 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672ba32a-2b99-4ef0-9894-6cdc585c337a admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:37:11.565145 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-672ba32a-2b99-4ef0-9894-6cdc585c337a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:37:11.594073 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672ba32a-2b99-4ef0-9894-6cdc585c337a admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:37:11.686296 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d9f5d7-59d1-45cf-8d35-8fbc5865a356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:37:11.687040 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d9f5d7-59d1-45cf-8d35-8fbc5865a356 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:11.687492 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d9f5d7-59d1-45cf-8d35-8fbc5865a356 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:11.688117 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d9f5d7-59d1-45cf-8d35-8fbc5865a356 admin admin] PATH_INFO: `/v3/projects/739bb819cf204184b577c0fcd9bd5ec1` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:11.689571 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d9f5d7-59d1-45cf-8d35-8fbc5865a356 admin admin] RBAC: Authorizing `identity:delete_project(project_id=739bb819cf204184b577c0fcd9bd5ec1)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:37:11.691896 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d9f5d7-59d1-45cf-8d35-8fbc5865a356 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:37:11.853388 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-a6d9f5d7-59d1-45cf-8d35-8fbc5865a356 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '739bb819cf204184b577c0fcd9bd5ec1', 'request_id': 'req-a6d9f5d7-59d1-45cf-8d35-8fbc5865a356'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:37:11.937453 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-a6d9f5d7-59d1-45cf-8d35-8fbc5865a356 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '739bb819cf204184b577c0fcd9bd5ec1', 'request_id': 'req-a6d9f5d7-59d1-45cf-8d35-8fbc5865a356'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:37:11.947407 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-a6d9f5d7-59d1-45cf-8d35-8fbc5865a356 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '739bb819cf204184b577c0fcd9bd5ec1', 'request_id': 'req-a6d9f5d7-59d1-45cf-8d35-8fbc5865a356'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:37:11.961089 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-a6d9f5d7-59d1-45cf-8d35-8fbc5865a356 admin admin] The token cache is being invalidate because project 739bb819cf204184b577c0fcd9bd5ec1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81732) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:37:11.961413 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-a6d9f5d7-59d1-45cf-8d35-8fbc5865a356 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:37:11.968340 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-a6d9f5d7-59d1-45cf-8d35-8fbc5865a356 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:37:11.970084 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1198/2421] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 24 17:37:11 2022] DELETE /identity/v3/projects/739bb819cf204184b577c0fcd9bd5ec1 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:37:11.978007 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f4d4a8-0640-46c8-966e-da2d54654fae None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:11.978797 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f4d4a8-0640-46c8-966e-da2d54654fae None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:11.978797 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f4d4a8-0640-46c8-966e-da2d54654fae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:12.004848 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-b2f4d4a8-0640-46c8-966e-da2d54654fae None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:37:12.016855 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-b2f4d4a8-0640-46c8-966e-da2d54654fae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:37:12.090869 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1224/2422] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:37:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:37:12.099320 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92faae43-bbfa-4ca4-bc85-998297229b78 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:12.099570 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92faae43-bbfa-4ca4-bc85-998297229b78 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:12.099894 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92faae43-bbfa-4ca4-bc85-998297229b78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:12.129637 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-92faae43-bbfa-4ca4-bc85-998297229b78 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:37:12.131282 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-92faae43-bbfa-4ca4-bc85-998297229b78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:37:12.139947 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1199/2423] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:37:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:37:12.147348 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f4d4a8-0640-46c8-966e-da2d54654fae None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:37:12.158879 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f4d4a8-0640-46c8-966e-da2d54654fae None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:37:12.164013 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc3d9a6-71e4-4f06-994d-ea94ade58120 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:37:12.164627 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc3d9a6-71e4-4f06-994d-ea94ade58120 admin admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:12.164909 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc3d9a6-71e4-4f06-994d-ea94ade58120 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:12.165181 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc3d9a6-71e4-4f06-994d-ea94ade58120 admin admin] PATH_INFO: `/v3/domains` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:12.165846 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc3d9a6-71e4-4f06-994d-ea94ade58120 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:37:12.166149 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc3d9a6-71e4-4f06-994d-ea94ade58120 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:37:12.167932 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc3d9a6-71e4-4f06-994d-ea94ade58120 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:37:12.180185 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1225/2424] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 24 17:37:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:37:12.184654 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92faae43-bbfa-4ca4-bc85-998297229b78 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:37:12.195903 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92faae43-bbfa-4ca4-bc85-998297229b78 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:37:12.200564 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c23d01b-d599-4fbe-bbe0-606c7df721d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:37:12.201182 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c23d01b-d599-4fbe-bbe0-606c7df721d7 admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:12.201469 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c23d01b-d599-4fbe-bbe0-606c7df721d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:12.201740 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c23d01b-d599-4fbe-bbe0-606c7df721d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:12.202378 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c23d01b-d599-4fbe-bbe0-606c7df721d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:37:12.204341 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c23d01b-d599-4fbe-bbe0-606c7df721d7 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:37:12.353496 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1200/2425] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 24 17:37:12 2022] POST /identity/v3/projects => generated 381 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:37:12.358707 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc3d9a6-71e4-4f06-994d-ea94ade58120 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:37:12.375432 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc3d9a6-71e4-4f06-994d-ea94ade58120 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:37:12.463078 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35eb21c-cb57-4b0f-8945-c389cc542492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:37:12.463701 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35eb21c-cb57-4b0f-8945-c389cc542492 admin admin] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:12.463983 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35eb21c-cb57-4b0f-8945-c389cc542492 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:12.464248 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35eb21c-cb57-4b0f-8945-c389cc542492 admin admin] PATH_INFO: `/v3/users` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:12.464883 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35eb21c-cb57-4b0f-8945-c389cc542492 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:37:12.466817 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35eb21c-cb57-4b0f-8945-c389cc542492 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:37:12.562652 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1226/2426] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 24 17:37:12 2022] POST /identity/v3/users => generated 291 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:37:12.567439 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c23d01b-d599-4fbe-bbe0-606c7df721d7 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:37:12.591086 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c23d01b-d599-4fbe-bbe0-606c7df721d7 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:37:12.597275 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739a8c98-4091-4a54-95a1-20bd6f17333b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:37:12.598162 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739a8c98-4091-4a54-95a1-20bd6f17333b admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:12.598723 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739a8c98-4091-4a54-95a1-20bd6f17333b admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:12.599482 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739a8c98-4091-4a54-95a1-20bd6f17333b admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:12.600689 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739a8c98-4091-4a54-95a1-20bd6f17333b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:37:12.603696 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739a8c98-4091-4a54-95a1-20bd6f17333b admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:37:12.620747 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1201/2427] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:37:12 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:37:12.629468 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35eb21c-cb57-4b0f-8945-c389cc542492 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:37:12.644400 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35eb21c-cb57-4b0f-8945-c389cc542492 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:37:12.651079 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e07915c-0b4d-453a-a86f-105f61628cbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:37:12.651930 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e07915c-0b4d-453a-a86f-105f61628cbe admin admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:12.652504 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e07915c-0b4d-453a-a86f-105f61628cbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:12.652953 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e07915c-0b4d-453a-a86f-105f61628cbe admin admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:12.653683 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e07915c-0b4d-453a-a86f-105f61628cbe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:37:12.655551 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e07915c-0b4d-453a-a86f-105f61628cbe admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:37:12.658172 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:37:12.671742 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1227/2428] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:37:12 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:37:12.676463 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739a8c98-4091-4a54-95a1-20bd6f17333b admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:37:12.690973 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739a8c98-4091-4a54-95a1-20bd6f17333b admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:37:12.698626 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355ff7cc-69c9-4f94-abc1-dd179776c099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:37:12.699591 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355ff7cc-69c9-4f94-abc1-dd179776c099 admin admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:12.699985 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355ff7cc-69c9-4f94-abc1-dd179776c099 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:12.700323 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355ff7cc-69c9-4f94-abc1-dd179776c099 admin admin] PATH_INFO: `/v3/projects/d728a8c77d964ad593ff8ba10e1bb228/users/8f2b78ba28c44053b47653e22bcaf897/roles/60dc7e16aee746d5a3e8aa27d7d8525c` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:12.703131 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:37:12.719354 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355ff7cc-69c9-4f94-abc1-dd179776c099 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d728a8c77d964ad593ff8ba10e1bb228, user_id=8f2b78ba28c44053b47653e22bcaf897, role_id=60dc7e16aee746d5a3e8aa27d7d8525c)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:37:12.722622 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355ff7cc-69c9-4f94-abc1-dd179776c099 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:37:12.813069 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1202/2429] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 24 17:37:12 2022] PUT /identity/v3/projects/d728a8c77d964ad593ff8ba10e1bb228/users/8f2b78ba28c44053b47653e22bcaf897/roles/60dc7e16aee746d5a3e8aa27d7d8525c => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:37:12.823330 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d1e644-4060-4f35-807e-d6697af9af01 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:12.823637 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d1e644-4060-4f35-807e-d6697af9af01 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:12.823914 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d1e644-4060-4f35-807e-d6697af9af01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:12.854428 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-85d1e644-4060-4f35-807e-d6697af9af01 None None] MFA Rules not processed for user `8f2b78ba28c44053b47653e22bcaf897`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:37:12.855684 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-85d1e644-4060-4f35-807e-d6697af9af01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:37:12.952777 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1228/2430] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:37:12 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:37:12.957469 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355ff7cc-69c9-4f94-abc1-dd179776c099 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:37:13.032945 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355ff7cc-69c9-4f94-abc1-dd179776c099 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:37:13.039128 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b5ab5a-44da-4329-810d-daadb418703a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:37:13.042003 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b5ab5a-44da-4329-810d-daadb418703a admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:13.042003 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b5ab5a-44da-4329-810d-daadb418703a admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:13.042003 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b5ab5a-44da-4329-810d-daadb418703a admin admin] PATH_INFO: `/v3/projects` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:13.042003 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b5ab5a-44da-4329-810d-daadb418703a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:37:13.042896 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b5ab5a-44da-4329-810d-daadb418703a admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:37:13.191610 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1203/2431] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 24 17:37:12 2022] POST /identity/v3/projects => generated 381 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:37:13.196605 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d1e644-4060-4f35-807e-d6697af9af01 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:37:13.209449 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d1e644-4060-4f35-807e-d6697af9af01 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:37:13.277567 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09dd4a0-bf81-4e83-b085-86ad689c2bf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:37:13.278202 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09dd4a0-bf81-4e83-b085-86ad689c2bf0 admin admin] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:13.278474 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09dd4a0-bf81-4e83-b085-86ad689c2bf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:13.278742 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09dd4a0-bf81-4e83-b085-86ad689c2bf0 admin admin] PATH_INFO: `/v3/users` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:13.279575 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09dd4a0-bf81-4e83-b085-86ad689c2bf0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:37:13.281539 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09dd4a0-bf81-4e83-b085-86ad689c2bf0 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:37:13.424537 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1229/2432] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 24 17:37:13 2022] POST /identity/v3/users => generated 297 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:37:13.429334 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b5ab5a-44da-4329-810d-daadb418703a admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:37:13.441981 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b5ab5a-44da-4329-810d-daadb418703a admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:37:13.447499 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1ccde6-6150-4a68-8335-e58473581a1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:37:13.450518 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1ccde6-6150-4a68-8335-e58473581a1e admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:13.450518 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1ccde6-6150-4a68-8335-e58473581a1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:13.450518 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1ccde6-6150-4a68-8335-e58473581a1e admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:13.450518 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1ccde6-6150-4a68-8335-e58473581a1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:37:13.451155 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1ccde6-6150-4a68-8335-e58473581a1e admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:37:13.467713 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1204/2433] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:37:13 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:37:13.472163 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09dd4a0-bf81-4e83-b085-86ad689c2bf0 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:37:13.482673 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09dd4a0-bf81-4e83-b085-86ad689c2bf0 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:37:13.487586 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bdaa2f-c47a-4773-98ad-a568a45a084c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:37:13.488250 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bdaa2f-c47a-4773-98ad-a568a45a084c admin admin] REQUEST_METHOD: `PUT` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:13.488532 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bdaa2f-c47a-4773-98ad-a568a45a084c admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:13.488811 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bdaa2f-c47a-4773-98ad-a568a45a084c admin admin] PATH_INFO: `/v3/projects/4555904e961d42369f2a0afd9127e35c/users/3cdd01f55284430493118f693a689e9d/roles/c5346bc54abf445f9900edf94264c850` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:13.505222 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bdaa2f-c47a-4773-98ad-a568a45a084c admin admin] RBAC: Authorizing `identity:create_grant(project_id=4555904e961d42369f2a0afd9127e35c, user_id=3cdd01f55284430493118f693a689e9d, role_id=c5346bc54abf445f9900edf94264c850)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:37:13.508077 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bdaa2f-c47a-4773-98ad-a568a45a084c admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:37:13.569023 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1230/2434] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 24 17:37:13 2022] PUT /identity/v3/projects/4555904e961d42369f2a0afd9127e35c/users/3cdd01f55284430493118f693a689e9d/roles/c5346bc54abf445f9900edf94264c850 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:37:13.573383 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1ccde6-6150-4a68-8335-e58473581a1e admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:37:13.585502 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1ccde6-6150-4a68-8335-e58473581a1e admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:37:13.669576 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81782969-1cad-4d3c-a446-b7656d9322cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:37:13.670590 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81782969-1cad-4d3c-a446-b7656d9322cf admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:13.671394 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81782969-1cad-4d3c-a446-b7656d9322cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:13.671892 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81782969-1cad-4d3c-a446-b7656d9322cf admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:13.673322 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81782969-1cad-4d3c-a446-b7656d9322cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:37:13.677023 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81782969-1cad-4d3c-a446-b7656d9322cf admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:37:13.690792 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1205/2435] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:37:13 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:37:13.699001 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bdaa2f-c47a-4773-98ad-a568a45a084c admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:37:13.713289 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bdaa2f-c47a-4773-98ad-a568a45a084c admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:37:13.719287 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a67795-8c15-4017-8d89-0b14ed83126c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:37:13.723007 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a67795-8c15-4017-8d89-0b14ed83126c admin admin] REQUEST_METHOD: `PUT` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:13.723007 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a67795-8c15-4017-8d89-0b14ed83126c admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:13.723007 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a67795-8c15-4017-8d89-0b14ed83126c admin admin] PATH_INFO: `/v3/projects/4555904e961d42369f2a0afd9127e35c/users/3cdd01f55284430493118f693a689e9d/roles/c5346bc54abf445f9900edf94264c850` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:13.723007 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a67795-8c15-4017-8d89-0b14ed83126c admin admin] RBAC: Authorizing `identity:create_grant(project_id=4555904e961d42369f2a0afd9127e35c, user_id=3cdd01f55284430493118f693a689e9d, role_id=c5346bc54abf445f9900edf94264c850)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:37:13.726148 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a67795-8c15-4017-8d89-0b14ed83126c admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:37:13.738990 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1231/2436] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 24 17:37:13 2022] PUT /identity/v3/projects/4555904e961d42369f2a0afd9127e35c/users/3cdd01f55284430493118f693a689e9d/roles/c5346bc54abf445f9900edf94264c850 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:37:13.757179 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667c2bdc-c753-47cc-a5aa-dab4d76d81a6 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:13.758569 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667c2bdc-c753-47cc-a5aa-dab4d76d81a6 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:13.759027 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667c2bdc-c753-47cc-a5aa-dab4d76d81a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:13.793386 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-667c2bdc-c753-47cc-a5aa-dab4d76d81a6 None None] MFA Rules not processed for user `3cdd01f55284430493118f693a689e9d`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:37:13.794741 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-667c2bdc-c753-47cc-a5aa-dab4d76d81a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:37:13.853500 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:37:13.900542 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1206/2437] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:37:13 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:37:13.919578 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a67795-8c15-4017-8d89-0b14ed83126c admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:37:13.921730 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-60a67795-8c15-4017-8d89-0b14ed83126c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:37:13.971911 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:37:14.001509 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a67795-8c15-4017-8d89-0b14ed83126c admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:37:14.008161 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3004c533-1f44-4669-b262-1d7e54bf9975 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be06246d47584b529150ceb98a5cfd95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:37:14.009018 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3004c533-1f44-4669-b262-1d7e54bf9975 service cinder] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:14.009471 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3004c533-1f44-4669-b262-1d7e54bf9975 service cinder] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:14.009930 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3004c533-1f44-4669-b262-1d7e54bf9975 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:14.028118 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3004c533-1f44-4669-b262-1d7e54bf9975 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:37:14.032722 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3004c533-1f44-4669-b262-1d7e54bf9975 service cinder] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:37:14.050012 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1232/2438] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:37:13 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:37:14.632535 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667c2bdc-c753-47cc-a5aa-dab4d76d81a6 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:37:14.701554 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667c2bdc-c753-47cc-a5aa-dab4d76d81a6 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:37:14.707193 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e7c224-0d2b-4c12-b5fb-d6a0899d0dea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:37:14.707840 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e7c224-0d2b-4c12-b5fb-d6a0899d0dea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:14.708122 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e7c224-0d2b-4c12-b5fb-d6a0899d0dea admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:14.708398 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e7c224-0d2b-4c12-b5fb-d6a0899d0dea admin admin] PATH_INFO: `/v3/users/8f2b78ba28c44053b47653e22bcaf897` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:14.709523 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e7c224-0d2b-4c12-b5fb-d6a0899d0dea admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f2b78ba28c44053b47653e22bcaf897)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:37:14.711508 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e7c224-0d2b-4c12-b5fb-d6a0899d0dea admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:37:14.929699 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-f1e7c224-0d2b-4c12-b5fb-d6a0899d0dea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f2b78ba28c44053b47653e22bcaf897'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:37:14.994618 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-f1e7c224-0d2b-4c12-b5fb-d6a0899d0dea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f2b78ba28c44053b47653e22bcaf897'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:37:14.995473 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-f1e7c224-0d2b-4c12-b5fb-d6a0899d0dea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f2b78ba28c44053b47653e22bcaf897'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:37:15.007148 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-f1e7c224-0d2b-4c12-b5fb-d6a0899d0dea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f2b78ba28c44053b47653e22bcaf897'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:37:15.007889 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-f1e7c224-0d2b-4c12-b5fb-d6a0899d0dea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f2b78ba28c44053b47653e22bcaf897'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:37:15.029644 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1207/2439] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 24 17:37:14 2022] DELETE /identity/v3/users/8f2b78ba28c44053b47653e22bcaf897 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:37:15.036046 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa72d4c-0849-4dc4-a55a-dd83f1cea827 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:15.036046 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa72d4c-0849-4dc4-a55a-dd83f1cea827 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:15.036046 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa72d4c-0849-4dc4-a55a-dd83f1cea827 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:15.057845 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-5aa72d4c-0849-4dc4-a55a-dd83f1cea827 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:37:15.059114 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-5aa72d4c-0849-4dc4-a55a-dd83f1cea827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:37:15.139719 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1233/2440] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:37:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:37:15.153698 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e7c224-0d2b-4c12-b5fb-d6a0899d0dea admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:37:15.156042 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-f1e7c224-0d2b-4c12-b5fb-d6a0899d0dea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:37:15.177426 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef01853d-759f-45b7-b6ab-b95388e67d4a None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:15.184301 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef01853d-759f-45b7-b6ab-b95388e67d4a None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:15.184301 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef01853d-759f-45b7-b6ab-b95388e67d4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:15.215128 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-ef01853d-759f-45b7-b6ab-b95388e67d4a None None] MFA Rules not processed for user `160ce86c89ea449da346d98d20ce2a9e`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:37:15.220765 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-ef01853d-759f-45b7-b6ab-b95388e67d4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:37:15.253554 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e7c224-0d2b-4c12-b5fb-d6a0899d0dea admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:37:15.257842 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f28b6b-b0d9-483d-84f4-8b684d1f4422 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e9ed4d47aef4140bfb2d3bb798cc559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:37:15.258061 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f28b6b-b0d9-483d-84f4-8b684d1f4422 service neutron] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:15.258182 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f28b6b-b0d9-483d-84f4-8b684d1f4422 service neutron] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:15.258457 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f28b6b-b0d9-483d-84f4-8b684d1f4422 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:15.272772 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f28b6b-b0d9-483d-84f4-8b684d1f4422 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:37:15.274237 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f28b6b-b0d9-483d-84f4-8b684d1f4422 service neutron] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:37:15.280477 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:37:15.288233 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1208/2441] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:37:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:37:15.296474 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1234/2442] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:37:15 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:37:15.696127 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f28b6b-b0d9-483d-84f4-8b684d1f4422 service neutron] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:37:15.705148 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f28b6b-b0d9-483d-84f4-8b684d1f4422 service neutron] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:37:15.710436 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c3f78b-9f15-4d35-9019-1f33f7e4789d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e9ed4d47aef4140bfb2d3bb798cc559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:37:15.711519 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c3f78b-9f15-4d35-9019-1f33f7e4789d service neutron] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:15.711920 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c3f78b-9f15-4d35-9019-1f33f7e4789d service neutron] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:15.712194 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c3f78b-9f15-4d35-9019-1f33f7e4789d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:15.715547 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-39c3f78b-9f15-4d35-9019-1f33f7e4789d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:37:15.730925 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c3f78b-9f15-4d35-9019-1f33f7e4789d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:37:15.732488 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c3f78b-9f15-4d35-9019-1f33f7e4789d service neutron] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:37:15.741897 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1209/2443] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:37:15 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:37:15.825747 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef01853d-759f-45b7-b6ab-b95388e67d4a None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:37:15.827850 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-ef01853d-759f-45b7-b6ab-b95388e67d4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:37:15.843475 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef01853d-759f-45b7-b6ab-b95388e67d4a None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:37:15.849042 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bc1ca9-929c-4e93-b0f0-205718900922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:37:15.849691 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bc1ca9-929c-4e93-b0f0-205718900922 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:15.850127 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bc1ca9-929c-4e93-b0f0-205718900922 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:15.862961 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bc1ca9-929c-4e93-b0f0-205718900922 admin admin] PATH_INFO: `/v3/projects/d728a8c77d964ad593ff8ba10e1bb228` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:15.862961 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bc1ca9-929c-4e93-b0f0-205718900922 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d728a8c77d964ad593ff8ba10e1bb228)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:37:15.866077 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bc1ca9-929c-4e93-b0f0-205718900922 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:37:16.039554 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-88bc1ca9-929c-4e93-b0f0-205718900922 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd728a8c77d964ad593ff8ba10e1bb228', 'request_id': 'req-88bc1ca9-929c-4e93-b0f0-205718900922'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:37:16.161377 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-88bc1ca9-929c-4e93-b0f0-205718900922 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd728a8c77d964ad593ff8ba10e1bb228', 'request_id': 'req-88bc1ca9-929c-4e93-b0f0-205718900922'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:37:16.169799 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-88bc1ca9-929c-4e93-b0f0-205718900922 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd728a8c77d964ad593ff8ba10e1bb228', 'request_id': 'req-88bc1ca9-929c-4e93-b0f0-205718900922'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:37:16.183580 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-88bc1ca9-929c-4e93-b0f0-205718900922 admin admin] The token cache is being invalidate because project d728a8c77d964ad593ff8ba10e1bb228 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81731) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:37:16.183874 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-88bc1ca9-929c-4e93-b0f0-205718900922 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:37:16.184560 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-88bc1ca9-929c-4e93-b0f0-205718900922 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:37:16.186072 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1235/2444] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 24 17:37:15 2022] DELETE /identity/v3/projects/d728a8c77d964ad593ff8ba10e1bb228 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:37:16.191557 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c3f78b-9f15-4d35-9019-1f33f7e4789d service neutron] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:37:16.220724 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c3f78b-9f15-4d35-9019-1f33f7e4789d service neutron] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:37:16.253662 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bc1ca9-929c-4e93-b0f0-205718900922 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:37:16.255266 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-88bc1ca9-929c-4e93-b0f0-205718900922 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:37:16.262305 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:37:16.342688 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162630e0-9a21-42d9-adf2-ea43e55a6ce9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:37:16.342688 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162630e0-9a21-42d9-adf2-ea43e55a6ce9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:16.342688 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162630e0-9a21-42d9-adf2-ea43e55a6ce9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:16.342688 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162630e0-9a21-42d9-adf2-ea43e55a6ce9 admin admin] PATH_INFO: `/v3/users/3cdd01f55284430493118f693a689e9d` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:16.344704 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162630e0-9a21-42d9-adf2-ea43e55a6ce9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3cdd01f55284430493118f693a689e9d)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:37:16.346891 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162630e0-9a21-42d9-adf2-ea43e55a6ce9 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:37:16.350793 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bc1ca9-929c-4e93-b0f0-205718900922 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:37:16.356657 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd985f42-9032-4c09-ad5c-8d369f7602de service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03fc77c6107e430f90f7b6f7cfd4ec02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:37:16.357283 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd985f42-9032-4c09-ad5c-8d369f7602de service placement] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:16.357568 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd985f42-9032-4c09-ad5c-8d369f7602de service placement] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:16.357849 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd985f42-9032-4c09-ad5c-8d369f7602de service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:16.359880 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-cd985f42-9032-4c09-ad5c-8d369f7602de service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:37:16.372922 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd985f42-9032-4c09-ad5c-8d369f7602de service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:37:16.382375 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd985f42-9032-4c09-ad5c-8d369f7602de service placement] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:37:16.385717 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1236/2445] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:37:16 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:37:16.579422 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd985f42-9032-4c09-ad5c-8d369f7602de service placement] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:37:16.581056 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-cd985f42-9032-4c09-ad5c-8d369f7602de service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:37:16.623176 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-162630e0-9a21-42d9-adf2-ea43e55a6ce9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3cdd01f55284430493118f693a689e9d'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:37:16.674272 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd985f42-9032-4c09-ad5c-8d369f7602de service placement] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:37:16.678651 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc28900-c8f5-4c93-a381-80cc33e7e6cb service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3553e522363046f6828fd61b2a0642ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:37:16.679296 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc28900-c8f5-4c93-a381-80cc33e7e6cb service glance] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:16.679577 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc28900-c8f5-4c93-a381-80cc33e7e6cb service glance] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:16.679851 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc28900-c8f5-4c93-a381-80cc33e7e6cb service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:16.681662 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-edc28900-c8f5-4c93-a381-80cc33e7e6cb service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:37:16.750131 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-162630e0-9a21-42d9-adf2-ea43e55a6ce9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3cdd01f55284430493118f693a689e9d'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:37:16.751824 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-162630e0-9a21-42d9-adf2-ea43e55a6ce9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3cdd01f55284430493118f693a689e9d'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:37:16.759772 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-162630e0-9a21-42d9-adf2-ea43e55a6ce9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3cdd01f55284430493118f693a689e9d'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:37:16.760444 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-162630e0-9a21-42d9-adf2-ea43e55a6ce9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3cdd01f55284430493118f693a689e9d'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:37:16.760995 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc28900-c8f5-4c93-a381-80cc33e7e6cb service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:37:16.762663 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc28900-c8f5-4c93-a381-80cc33e7e6cb service glance] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:37:16.776148 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1237/2446] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:37:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:37:16.781392 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1210/2447] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 24 17:37:16 2022] DELETE /identity/v3/users/3cdd01f55284430493118f693a689e9d => generated 0 bytes in 591 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:37:17.028579 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc28900-c8f5-4c93-a381-80cc33e7e6cb service glance] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:37:17.030989 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-edc28900-c8f5-4c93-a381-80cc33e7e6cb service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:37:17.101915 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc28900-c8f5-4c93-a381-80cc33e7e6cb service glance] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:37:17.163526 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99eae86e-63ed-41ac-b045-41bf5a944275 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e9ed4d47aef4140bfb2d3bb798cc559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:37:17.164150 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eae86e-63ed-41ac-b045-41bf5a944275 service neutron] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:17.164432 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eae86e-63ed-41ac-b045-41bf5a944275 service neutron] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:17.164707 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eae86e-63ed-41ac-b045-41bf5a944275 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:17.166549 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-99eae86e-63ed-41ac-b045-41bf5a944275 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:37:17.177552 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99eae86e-63ed-41ac-b045-41bf5a944275 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:37:17.179151 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99eae86e-63ed-41ac-b045-41bf5a944275 service neutron] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:37:17.188349 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1238/2448] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:37:17 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:37:17.265463 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162630e0-9a21-42d9-adf2-ea43e55a6ce9 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:37:17.267013 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-162630e0-9a21-42d9-adf2-ea43e55a6ce9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:37:17.276226 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:37:17.281026 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162630e0-9a21-42d9-adf2-ea43e55a6ce9 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:37:17.285401 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99eae86e-63ed-41ac-b045-41bf5a944275 service neutron] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:37:17.287024 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-99eae86e-63ed-41ac-b045-41bf5a944275 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:37:17.288945 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d23328-542f-4273-991c-9f122089d1c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e9ed4d47aef4140bfb2d3bb798cc559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:37:17.290385 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d23328-542f-4273-991c-9f122089d1c4 service neutron] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:17.290884 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d23328-542f-4273-991c-9f122089d1c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:17.291410 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d23328-542f-4273-991c-9f122089d1c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:17.294387 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-92d23328-542f-4273-991c-9f122089d1c4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:37:17.373506 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99eae86e-63ed-41ac-b045-41bf5a944275 service neutron] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:37:17.379506 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0f17bf-c7dd-4354-b94c-810145670fd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:37:17.380160 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0f17bf-c7dd-4354-b94c-810145670fd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:17.380486 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0f17bf-c7dd-4354-b94c-810145670fd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:17.380786 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0f17bf-c7dd-4354-b94c-810145670fd9 admin admin] PATH_INFO: `/v3/projects/4555904e961d42369f2a0afd9127e35c` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:17.381849 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0f17bf-c7dd-4354-b94c-810145670fd9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4555904e961d42369f2a0afd9127e35c)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:37:17.384044 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0f17bf-c7dd-4354-b94c-810145670fd9 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:37:17.400593 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:37:17.427381 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d23328-542f-4273-991c-9f122089d1c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:37:17.429144 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d23328-542f-4273-991c-9f122089d1c4 service neutron] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:37:17.442492 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1211/2449] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:37:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:37:17.534159 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-3c0f17bf-c7dd-4354-b94c-810145670fd9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4555904e961d42369f2a0afd9127e35c', 'request_id': 'req-3c0f17bf-c7dd-4354-b94c-810145670fd9'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:37:17.630360 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-3c0f17bf-c7dd-4354-b94c-810145670fd9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4555904e961d42369f2a0afd9127e35c', 'request_id': 'req-3c0f17bf-c7dd-4354-b94c-810145670fd9'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:37:17.640486 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-3c0f17bf-c7dd-4354-b94c-810145670fd9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4555904e961d42369f2a0afd9127e35c', 'request_id': 'req-3c0f17bf-c7dd-4354-b94c-810145670fd9'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:37:17.654188 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-3c0f17bf-c7dd-4354-b94c-810145670fd9 admin admin] The token cache is being invalidate because project 4555904e961d42369f2a0afd9127e35c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81731) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:37:17.654551 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-3c0f17bf-c7dd-4354-b94c-810145670fd9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:37:17.655326 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-3c0f17bf-c7dd-4354-b94c-810145670fd9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:37:17.657272 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1239/2450] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 24 17:37:17 2022] DELETE /identity/v3/projects/4555904e961d42369f2a0afd9127e35c => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:37:17.663851 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fed4af3-ec78-437d-bee4-58f3b1cd3f27 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:17.664147 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fed4af3-ec78-437d-bee4-58f3b1cd3f27 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:17.664432 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fed4af3-ec78-437d-bee4-58f3b1cd3f27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:17.691986 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-3fed4af3-ec78-437d-bee4-58f3b1cd3f27 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:37:17.693295 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-3fed4af3-ec78-437d-bee4-58f3b1cd3f27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:37:17.760280 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1212/2451] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:37:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:37:17.769860 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a6a960-02a0-43ec-b91b-d696b2e3a5cc None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:17.770280 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a6a960-02a0-43ec-b91b-d696b2e3a5cc None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:17.770672 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a6a960-02a0-43ec-b91b-d696b2e3a5cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:17.779092 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:37:17.792828 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-88a6a960-02a0-43ec-b91b-d696b2e3a5cc None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:37:17.794175 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-88a6a960-02a0-43ec-b91b-d696b2e3a5cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:37:17.803338 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1240/2452] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:37:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:37:17.807650 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fed4af3-ec78-437d-bee4-58f3b1cd3f27 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:37:17.819171 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fed4af3-ec78-437d-bee4-58f3b1cd3f27 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:37:17.825298 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1abcbe6-6659-4480-9603-bb9e34128f77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:37:17.826145 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1abcbe6-6659-4480-9603-bb9e34128f77 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:17.826434 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1abcbe6-6659-4480-9603-bb9e34128f77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:17.826720 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1abcbe6-6659-4480-9603-bb9e34128f77 admin admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:17.827419 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1abcbe6-6659-4480-9603-bb9e34128f77 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:37:17.827724 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1abcbe6-6659-4480-9603-bb9e34128f77 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:37:17.829381 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1abcbe6-6659-4480-9603-bb9e34128f77 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:37:17.843661 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1213/2453] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 24 17:37:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:37:17.849236 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a6a960-02a0-43ec-b91b-d696b2e3a5cc None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:37:17.859302 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a6a960-02a0-43ec-b91b-d696b2e3a5cc None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:37:17.864658 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db262a66-5c61-4018-9b9e-f906e18b2188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:37:17.865026 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db262a66-5c61-4018-9b9e-f906e18b2188 admin admin] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:17.865425 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db262a66-5c61-4018-9b9e-f906e18b2188 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:17.865709 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db262a66-5c61-4018-9b9e-f906e18b2188 admin admin] PATH_INFO: `/v3/projects` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:17.866417 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db262a66-5c61-4018-9b9e-f906e18b2188 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:37:17.868551 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db262a66-5c61-4018-9b9e-f906e18b2188 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:37:18.017519 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1241/2454] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 24 17:37:17 2022] POST /identity/v3/projects => generated 383 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:37:18.022083 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1abcbe6-6659-4480-9603-bb9e34128f77 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:37:18.044163 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1abcbe6-6659-4480-9603-bb9e34128f77 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:37:18.141008 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8c8357-0436-409d-a05d-52c340b8fea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:37:18.142025 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8c8357-0436-409d-a05d-52c340b8fea9 admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:18.142511 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8c8357-0436-409d-a05d-52c340b8fea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:18.143057 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8c8357-0436-409d-a05d-52c340b8fea9 admin admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:18.144962 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8c8357-0436-409d-a05d-52c340b8fea9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:37:18.149908 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8c8357-0436-409d-a05d-52c340b8fea9 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:37:18.299705 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1214/2455] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 24 17:37:18 2022] POST /identity/v3/users => generated 292 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:37:18.305546 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db262a66-5c61-4018-9b9e-f906e18b2188 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:37:18.317354 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db262a66-5c61-4018-9b9e-f906e18b2188 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:37:18.322627 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbff889-32e2-4429-9d36-e10008ad9ab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:37:18.323294 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbff889-32e2-4429-9d36-e10008ad9ab5 admin admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:18.323571 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbff889-32e2-4429-9d36-e10008ad9ab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:18.323844 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbff889-32e2-4429-9d36-e10008ad9ab5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:18.324505 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbff889-32e2-4429-9d36-e10008ad9ab5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:37:18.326136 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbff889-32e2-4429-9d36-e10008ad9ab5 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:37:18.337592 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1242/2456] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:37:18 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:37:18.342062 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8c8357-0436-409d-a05d-52c340b8fea9 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:37:18.357548 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8c8357-0436-409d-a05d-52c340b8fea9 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:37:18.363791 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574444ef-b8dc-4753-ad2d-71806b4ee4e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:37:18.364557 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574444ef-b8dc-4753-ad2d-71806b4ee4e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:18.364848 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574444ef-b8dc-4753-ad2d-71806b4ee4e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:18.365110 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574444ef-b8dc-4753-ad2d-71806b4ee4e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:18.365757 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574444ef-b8dc-4753-ad2d-71806b4ee4e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:37:18.367398 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574444ef-b8dc-4753-ad2d-71806b4ee4e7 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:37:18.383788 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1215/2457] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:37:18 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:37:18.388038 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbff889-32e2-4429-9d36-e10008ad9ab5 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:37:18.398147 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbff889-32e2-4429-9d36-e10008ad9ab5 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:37:18.403349 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e1d554-aeea-4b23-b882-a4096d30d85e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:37:18.404124 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e1d554-aeea-4b23-b882-a4096d30d85e admin admin] REQUEST_METHOD: `PUT` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:18.404294 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e1d554-aeea-4b23-b882-a4096d30d85e admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:18.404564 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e1d554-aeea-4b23-b882-a4096d30d85e admin admin] PATH_INFO: `/v3/projects/b50bd896cd444d9b9fb619ac99c6cf22/users/26600a852cdb44bfbd95c6d4ab72a9e5/roles/60dc7e16aee746d5a3e8aa27d7d8525c` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:18.420988 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e1d554-aeea-4b23-b882-a4096d30d85e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b50bd896cd444d9b9fb619ac99c6cf22, user_id=26600a852cdb44bfbd95c6d4ab72a9e5, role_id=60dc7e16aee746d5a3e8aa27d7d8525c)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:37:18.423730 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e1d554-aeea-4b23-b882-a4096d30d85e admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:37:18.539692 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1243/2458] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 24 17:37:18 2022] PUT /identity/v3/projects/b50bd896cd444d9b9fb619ac99c6cf22/users/26600a852cdb44bfbd95c6d4ab72a9e5/roles/60dc7e16aee746d5a3e8aa27d7d8525c => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:37:18.552535 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fa35f2-b14c-4117-aed9-9e43a5ccf638 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:18.552832 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fa35f2-b14c-4117-aed9-9e43a5ccf638 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:18.553336 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fa35f2-b14c-4117-aed9-9e43a5ccf638 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:18.584559 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-e8fa35f2-b14c-4117-aed9-9e43a5ccf638 None None] MFA Rules not processed for user `26600a852cdb44bfbd95c6d4ab72a9e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:37:18.585775 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-e8fa35f2-b14c-4117-aed9-9e43a5ccf638 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:37:18.640145 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:37:18.671304 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1216/2459] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:37:18 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:37:18.683842 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e1d554-aeea-4b23-b882-a4096d30d85e admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:37:18.685849 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-14e1d554-aeea-4b23-b882-a4096d30d85e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:37:18.749261 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e1d554-aeea-4b23-b882-a4096d30d85e admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:37:18.753764 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71188cd-6382-424d-8533-afa5244ac04d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be06246d47584b529150ceb98a5cfd95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:37:18.754379 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71188cd-6382-424d-8533-afa5244ac04d service cinder] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:18.754660 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71188cd-6382-424d-8533-afa5244ac04d service cinder] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:18.754961 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71188cd-6382-424d-8533-afa5244ac04d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:18.764138 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71188cd-6382-424d-8533-afa5244ac04d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:37:18.765698 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71188cd-6382-424d-8533-afa5244ac04d service cinder] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:37:18.776406 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1244/2460] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:37:18 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:37:18.796872 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fa35f2-b14c-4117-aed9-9e43a5ccf638 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:37:18.873469 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fa35f2-b14c-4117-aed9-9e43a5ccf638 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:37:18.878208 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f44ab4c-5e03-4d67-adc6-3a7fe81c1f96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:37:18.879416 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f44ab4c-5e03-4d67-adc6-3a7fe81c1f96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:18.879700 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f44ab4c-5e03-4d67-adc6-3a7fe81c1f96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:18.879976 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f44ab4c-5e03-4d67-adc6-3a7fe81c1f96 admin admin] PATH_INFO: `/v3/users/26600a852cdb44bfbd95c6d4ab72a9e5` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:18.881118 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f44ab4c-5e03-4d67-adc6-3a7fe81c1f96 admin admin] RBAC: Authorizing `identity:delete_user(user_id=26600a852cdb44bfbd95c6d4ab72a9e5)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:37:18.883123 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f44ab4c-5e03-4d67-adc6-3a7fe81c1f96 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:37:19.171291 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-8f44ab4c-5e03-4d67-adc6-3a7fe81c1f96 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26600a852cdb44bfbd95c6d4ab72a9e5'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:37:19.280552 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-8f44ab4c-5e03-4d67-adc6-3a7fe81c1f96 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26600a852cdb44bfbd95c6d4ab72a9e5'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:37:19.281582 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-8f44ab4c-5e03-4d67-adc6-3a7fe81c1f96 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26600a852cdb44bfbd95c6d4ab72a9e5'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:37:19.297199 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-8f44ab4c-5e03-4d67-adc6-3a7fe81c1f96 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26600a852cdb44bfbd95c6d4ab72a9e5'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:37:19.297199 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-8f44ab4c-5e03-4d67-adc6-3a7fe81c1f96 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26600a852cdb44bfbd95c6d4ab72a9e5'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:37:19.317216 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1217/2461] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 24 17:37:18 2022] DELETE /identity/v3/users/26600a852cdb44bfbd95c6d4ab72a9e5 => generated 0 bytes in 521 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:37:19.322133 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703de314-e727-465a-bc9c-b83440d7b720 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:19.326422 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703de314-e727-465a-bc9c-b83440d7b720 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:19.326422 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703de314-e727-465a-bc9c-b83440d7b720 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:19.343385 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-703de314-e727-465a-bc9c-b83440d7b720 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:37:19.344802 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-703de314-e727-465a-bc9c-b83440d7b720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:37:19.420310 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1245/2462] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:37:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:37:19.430993 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f44ab4c-5e03-4d67-adc6-3a7fe81c1f96 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:37:19.432524 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-8f44ab4c-5e03-4d67-adc6-3a7fe81c1f96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:37:19.504697 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f44ab4c-5e03-4d67-adc6-3a7fe81c1f96 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:37:19.508419 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5beecc50-371a-4f67-af7e-2bbe0c1aa25b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e9ed4d47aef4140bfb2d3bb798cc559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:37:19.508980 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5beecc50-371a-4f67-af7e-2bbe0c1aa25b service neutron] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:19.509259 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5beecc50-371a-4f67-af7e-2bbe0c1aa25b service neutron] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:19.509572 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5beecc50-371a-4f67-af7e-2bbe0c1aa25b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:19.521625 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5beecc50-371a-4f67-af7e-2bbe0c1aa25b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:37:19.523324 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5beecc50-371a-4f67-af7e-2bbe0c1aa25b service neutron] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:37:19.538137 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1218/2463] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:37:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:37:20.254982 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703de314-e727-465a-bc9c-b83440d7b720 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:37:20.256535 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-703de314-e727-465a-bc9c-b83440d7b720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:37:20.268396 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703de314-e727-465a-bc9c-b83440d7b720 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:37:20.273991 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2fe0bd-aee1-4a74-a312-c6b136109305 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:37:20.274804 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2fe0bd-aee1-4a74-a312-c6b136109305 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:20.275216 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2fe0bd-aee1-4a74-a312-c6b136109305 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:20.275539 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2fe0bd-aee1-4a74-a312-c6b136109305 admin admin] PATH_INFO: `/v3/projects/b50bd896cd444d9b9fb619ac99c6cf22` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:20.276812 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2fe0bd-aee1-4a74-a312-c6b136109305 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b50bd896cd444d9b9fb619ac99c6cf22)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:37:20.279242 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2fe0bd-aee1-4a74-a312-c6b136109305 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:37:20.433906 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:37:20.464585 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-5f2fe0bd-aee1-4a74-a312-c6b136109305 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b50bd896cd444d9b9fb619ac99c6cf22', 'request_id': 'req-5f2fe0bd-aee1-4a74-a312-c6b136109305'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:37:20.573604 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-5f2fe0bd-aee1-4a74-a312-c6b136109305 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b50bd896cd444d9b9fb619ac99c6cf22', 'request_id': 'req-5f2fe0bd-aee1-4a74-a312-c6b136109305'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:37:20.584156 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-5f2fe0bd-aee1-4a74-a312-c6b136109305 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b50bd896cd444d9b9fb619ac99c6cf22', 'request_id': 'req-5f2fe0bd-aee1-4a74-a312-c6b136109305'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:37:20.601943 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-5f2fe0bd-aee1-4a74-a312-c6b136109305 admin admin] The token cache is being invalidate because project b50bd896cd444d9b9fb619ac99c6cf22 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81731) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:37:20.602389 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-5f2fe0bd-aee1-4a74-a312-c6b136109305 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:37:20.603166 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-5f2fe0bd-aee1-4a74-a312-c6b136109305 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:37:20.604989 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1246/2464] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 24 17:37:20 2022] DELETE /identity/v3/projects/b50bd896cd444d9b9fb619ac99c6cf22 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:37:20.611019 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6799aba4-9601-4eb5-8e7f-69b6c03382a7 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:20.611517 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6799aba4-9601-4eb5-8e7f-69b6c03382a7 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:20.615438 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6799aba4-9601-4eb5-8e7f-69b6c03382a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:20.634365 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-6799aba4-9601-4eb5-8e7f-69b6c03382a7 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:37:20.635813 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-6799aba4-9601-4eb5-8e7f-69b6c03382a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:37:20.640762 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:37:20.641103 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2fe0bd-aee1-4a74-a312-c6b136109305 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:37:20.645950 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-5f2fe0bd-aee1-4a74-a312-c6b136109305 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:37:20.728972 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1219/2465] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:37:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:37:20.736367 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee1c845-9b99-4aff-b148-f5c164ec0d05 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:20.737122 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee1c845-9b99-4aff-b148-f5c164ec0d05 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:20.737600 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee1c845-9b99-4aff-b148-f5c164ec0d05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:20.762164 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2fe0bd-aee1-4a74-a312-c6b136109305 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:37:20.764182 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-6ee1c845-9b99-4aff-b148-f5c164ec0d05 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:37:20.766291 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-6ee1c845-9b99-4aff-b148-f5c164ec0d05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:37:20.779563 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1220/2466] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:37:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:37:20.785441 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee1c845-9b99-4aff-b148-f5c164ec0d05 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:37:20.791243 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6e33d4-e9fb-4383-a06d-12ccc2189ee7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557aca9255cb4bbb8af52e796c5c5177', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:37:20.791243 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6e33d4-e9fb-4383-a06d-12ccc2189ee7 service nova] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:20.791243 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6e33d4-e9fb-4383-a06d-12ccc2189ee7 service nova] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:20.791243 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6e33d4-e9fb-4383-a06d-12ccc2189ee7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:20.796330 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-3d6e33d4-e9fb-4383-a06d-12ccc2189ee7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:37:20.797681 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee1c845-9b99-4aff-b148-f5c164ec0d05 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:37:20.803148 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caff7db1-6e66-4015-bf1f-79dfc9a7f12e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:37:20.803681 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caff7db1-6e66-4015-bf1f-79dfc9a7f12e admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:20.803970 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caff7db1-6e66-4015-bf1f-79dfc9a7f12e admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:20.804291 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caff7db1-6e66-4015-bf1f-79dfc9a7f12e admin admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:20.804962 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caff7db1-6e66-4015-bf1f-79dfc9a7f12e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:37:20.805267 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caff7db1-6e66-4015-bf1f-79dfc9a7f12e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:37:20.811369 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caff7db1-6e66-4015-bf1f-79dfc9a7f12e admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:37:20.838250 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1221/2467] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 24 17:37:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:37:20.843414 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caff7db1-6e66-4015-bf1f-79dfc9a7f12e admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:37:20.877018 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caff7db1-6e66-4015-bf1f-79dfc9a7f12e admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:37:20.884886 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796b77bd-b061-4486-b7e2-e306410f6f4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:37:20.886308 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796b77bd-b061-4486-b7e2-e306410f6f4a admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:20.886889 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796b77bd-b061-4486-b7e2-e306410f6f4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:20.887471 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796b77bd-b061-4486-b7e2-e306410f6f4a admin admin] PATH_INFO: `/v3/projects` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:20.888541 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796b77bd-b061-4486-b7e2-e306410f6f4a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:37:20.892873 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796b77bd-b061-4486-b7e2-e306410f6f4a admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:37:20.927861 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6e33d4-e9fb-4383-a06d-12ccc2189ee7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:37:20.929474 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6e33d4-e9fb-4383-a06d-12ccc2189ee7 service nova] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:37:20.958262 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1247/2468] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:37:20 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:37:20.972480 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d178d3-f888-4d08-96c2-a5d9a2691108 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:20.972480 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d178d3-f888-4d08-96c2-a5d9a2691108 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:20.972480 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d178d3-f888-4d08-96c2-a5d9a2691108 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:20.988061 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-73d178d3-f888-4d08-96c2-a5d9a2691108 None None] MFA Rules not processed for user `0ccc6723f7514e52863d102de3839951`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:37:20.989445 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-73d178d3-f888-4d08-96c2-a5d9a2691108 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:37:21.005077 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1222/2469] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 24 17:37:20 2022] POST /identity/v3/projects => generated 361 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:37:21.010027 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796b77bd-b061-4486-b7e2-e306410f6f4a admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:37:21.075386 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1248/2470] 10.222.0.18 () {58 vars in 1009 bytes} [Wed Aug 24 17:37:20 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:37:21.090819 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d178d3-f888-4d08-96c2-a5d9a2691108 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:37:21.093596 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-73d178d3-f888-4d08-96c2-a5d9a2691108 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:37:21.117655 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796b77bd-b061-4486-b7e2-e306410f6f4a admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:37:21.124230 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7accee-e973-4484-a113-62dd4cb3057a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:37:21.125535 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7accee-e973-4484-a113-62dd4cb3057a admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:21.125911 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7accee-e973-4484-a113-62dd4cb3057a admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:21.126257 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7accee-e973-4484-a113-62dd4cb3057a admin admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:21.127181 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7accee-e973-4484-a113-62dd4cb3057a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:37:21.129347 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7accee-e973-4484-a113-62dd4cb3057a admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:37:21.187502 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d178d3-f888-4d08-96c2-a5d9a2691108 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:37:21.252599 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8090ae6-b1f2-4268-9316-aef62089e8cf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef8caa353be34499a58cd25b72ee4032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:37:21.253130 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8090ae6-b1f2-4268-9316-aef62089e8cf service swift] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:21.256218 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8090ae6-b1f2-4268-9316-aef62089e8cf service swift] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:21.256218 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8090ae6-b1f2-4268-9316-aef62089e8cf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:21.311998 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8090ae6-b1f2-4268-9316-aef62089e8cf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:37:21.313712 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8090ae6-b1f2-4268-9316-aef62089e8cf service swift] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:37:21.318868 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1223/2471] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 24 17:37:21 2022] POST /identity/v3/users => generated 281 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:37:21.320583 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7accee-e973-4484-a113-62dd4cb3057a admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:37:21.324193 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1249/2472] 10.222.0.18 () {60 vars in 1384 bytes} [Wed Aug 24 17:37:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 24 17:37:21.334135 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7accee-e973-4484-a113-62dd4cb3057a admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:37:21.340089 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3652313-472f-4a8e-a81d-e77f1f6ae1da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:37:21.340445 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3652313-472f-4a8e-a81d-e77f1f6ae1da admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:21.340746 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3652313-472f-4a8e-a81d-e77f1f6ae1da admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:21.341088 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3652313-472f-4a8e-a81d-e77f1f6ae1da admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:21.341830 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3652313-472f-4a8e-a81d-e77f1f6ae1da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:37:21.348330 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3652313-472f-4a8e-a81d-e77f1f6ae1da admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:37:21.356976 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1224/2473] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:37:21 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:37:21.361432 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8090ae6-b1f2-4268-9316-aef62089e8cf service swift] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:37:21.373711 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8090ae6-b1f2-4268-9316-aef62089e8cf service swift] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:37:21.383832 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14cd1c2-0af9-4ec7-afdd-2324ec0bfc88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:37:21.384480 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14cd1c2-0af9-4ec7-afdd-2324ec0bfc88 admin admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:21.384751 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14cd1c2-0af9-4ec7-afdd-2324ec0bfc88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:21.385020 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14cd1c2-0af9-4ec7-afdd-2324ec0bfc88 admin admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:21.385674 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14cd1c2-0af9-4ec7-afdd-2324ec0bfc88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:37:21.387505 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14cd1c2-0af9-4ec7-afdd-2324ec0bfc88 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:37:21.401879 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1250/2474] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:37:21 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:37:21.403669 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3652313-472f-4a8e-a81d-e77f1f6ae1da admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:37:21.415206 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3652313-472f-4a8e-a81d-e77f1f6ae1da admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:37:21.421507 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b077f10-638b-4f40-85c3-140694bd17f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:37:21.422573 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b077f10-638b-4f40-85c3-140694bd17f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:21.422947 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b077f10-638b-4f40-85c3-140694bd17f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:21.423252 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b077f10-638b-4f40-85c3-140694bd17f2 admin admin] PATH_INFO: `/v3/projects/1ba1db4eb9774440b0acc477e68a81c2/users/2c246be686f74531976427a0c055d3e4/roles/60dc7e16aee746d5a3e8aa27d7d8525c` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:21.440458 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b077f10-638b-4f40-85c3-140694bd17f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ba1db4eb9774440b0acc477e68a81c2, user_id=2c246be686f74531976427a0c055d3e4, role_id=60dc7e16aee746d5a3e8aa27d7d8525c)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:37:21.444084 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b077f10-638b-4f40-85c3-140694bd17f2 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:37:21.591370 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1225/2475] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 24 17:37:21 2022] PUT /identity/v3/projects/1ba1db4eb9774440b0acc477e68a81c2/users/2c246be686f74531976427a0c055d3e4/roles/60dc7e16aee746d5a3e8aa27d7d8525c => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:37:21.598849 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c74d138-0c4a-472a-b2ae-182e3b27b2eb None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:21.600754 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c74d138-0c4a-472a-b2ae-182e3b27b2eb None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:21.602207 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c74d138-0c4a-472a-b2ae-182e3b27b2eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:21.612988 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:37:21.696821 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-4c74d138-0c4a-472a-b2ae-182e3b27b2eb None None] MFA Rules not processed for user `2c246be686f74531976427a0c055d3e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:37:21.699125 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-4c74d138-0c4a-472a-b2ae-182e3b27b2eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:37:21.944246 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1251/2476] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:37:21 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 353 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:37:22.000093 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b077f10-638b-4f40-85c3-140694bd17f2 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:37:22.003158 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-2b077f10-638b-4f40-85c3-140694bd17f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:37:22.071597 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:37:22.136323 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b077f10-638b-4f40-85c3-140694bd17f2 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:37:22.140800 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad200c50-6d78-4f8a-9147-2160967cade9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be06246d47584b529150ceb98a5cfd95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:37:22.141599 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad200c50-6d78-4f8a-9147-2160967cade9 service cinder] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:22.141883 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad200c50-6d78-4f8a-9147-2160967cade9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:22.142149 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad200c50-6d78-4f8a-9147-2160967cade9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:22.154996 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad200c50-6d78-4f8a-9147-2160967cade9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:37:22.156744 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad200c50-6d78-4f8a-9147-2160967cade9 service cinder] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:37:22.175783 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1226/2477] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:37:21 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:37:22.195412 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c74d138-0c4a-472a-b2ae-182e3b27b2eb None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:37:22.318419 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c74d138-0c4a-472a-b2ae-182e3b27b2eb None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:37:22.330323 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2271df66-1bd8-4a62-8efb-64859fbaadaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:37:22.330323 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2271df66-1bd8-4a62-8efb-64859fbaadaf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:22.330323 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2271df66-1bd8-4a62-8efb-64859fbaadaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:22.330323 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2271df66-1bd8-4a62-8efb-64859fbaadaf admin admin] PATH_INFO: `/v3/users/2c246be686f74531976427a0c055d3e4` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:22.331038 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2271df66-1bd8-4a62-8efb-64859fbaadaf admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c246be686f74531976427a0c055d3e4)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:37:22.331038 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2271df66-1bd8-4a62-8efb-64859fbaadaf admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:37:22.660467 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-2271df66-1bd8-4a62-8efb-64859fbaadaf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c246be686f74531976427a0c055d3e4'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:37:22.674023 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-2271df66-1bd8-4a62-8efb-64859fbaadaf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c246be686f74531976427a0c055d3e4'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:37:22.770506 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-2271df66-1bd8-4a62-8efb-64859fbaadaf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c246be686f74531976427a0c055d3e4'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:37:22.771159 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-2271df66-1bd8-4a62-8efb-64859fbaadaf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c246be686f74531976427a0c055d3e4'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:37:22.778892 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-2271df66-1bd8-4a62-8efb-64859fbaadaf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c246be686f74531976427a0c055d3e4'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:37:22.785085 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1252/2478] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 24 17:37:22 2022] DELETE /identity/v3/users/2c246be686f74531976427a0c055d3e4 => generated 0 bytes in 591 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:37:22.790504 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6616fed1-f137-4913-b5ec-4f0cd03017e5 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:22.798692 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6616fed1-f137-4913-b5ec-4f0cd03017e5 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:22.798692 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6616fed1-f137-4913-b5ec-4f0cd03017e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:22.813679 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-6616fed1-f137-4913-b5ec-4f0cd03017e5 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:37:22.814893 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-6616fed1-f137-4913-b5ec-4f0cd03017e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:37:22.876932 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1227/2479] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:37:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:37:22.888035 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2271df66-1bd8-4a62-8efb-64859fbaadaf admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:37:22.889467 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-2271df66-1bd8-4a62-8efb-64859fbaadaf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:37:22.938713 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:37:22.951453 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2271df66-1bd8-4a62-8efb-64859fbaadaf admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:37:22.961398 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f860a49a-0672-4976-8c81-8c8ea94dcfbd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e9ed4d47aef4140bfb2d3bb798cc559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:37:22.962076 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f860a49a-0672-4976-8c81-8c8ea94dcfbd service neutron] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:22.962365 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f860a49a-0672-4976-8c81-8c8ea94dcfbd service neutron] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:22.962666 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f860a49a-0672-4976-8c81-8c8ea94dcfbd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:22.974713 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f860a49a-0672-4976-8c81-8c8ea94dcfbd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:37:22.976859 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f860a49a-0672-4976-8c81-8c8ea94dcfbd service neutron] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:37:22.987801 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1253/2480] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:37:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:37:23.000388 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:37:23.560761 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6616fed1-f137-4913-b5ec-4f0cd03017e5 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:37:23.562234 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-6616fed1-f137-4913-b5ec-4f0cd03017e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:37:23.577681 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6616fed1-f137-4913-b5ec-4f0cd03017e5 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:37:23.590573 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcce347-3589-47df-a110-5c7ab070b7c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:37:23.592160 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcce347-3589-47df-a110-5c7ab070b7c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:23.592665 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcce347-3589-47df-a110-5c7ab070b7c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:23.593223 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcce347-3589-47df-a110-5c7ab070b7c3 admin admin] PATH_INFO: `/v3/projects/1ba1db4eb9774440b0acc477e68a81c2` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:23.596117 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcce347-3589-47df-a110-5c7ab070b7c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ba1db4eb9774440b0acc477e68a81c2)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:37:23.600378 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcce347-3589-47df-a110-5c7ab070b7c3 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:37:23.765358 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-6fcce347-3589-47df-a110-5c7ab070b7c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ba1db4eb9774440b0acc477e68a81c2', 'request_id': 'req-6fcce347-3589-47df-a110-5c7ab070b7c3'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:37:23.857248 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-6fcce347-3589-47df-a110-5c7ab070b7c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ba1db4eb9774440b0acc477e68a81c2', 'request_id': 'req-6fcce347-3589-47df-a110-5c7ab070b7c3'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:37:23.864828 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-6fcce347-3589-47df-a110-5c7ab070b7c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ba1db4eb9774440b0acc477e68a81c2', 'request_id': 'req-6fcce347-3589-47df-a110-5c7ab070b7c3'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:37:23.875715 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-6fcce347-3589-47df-a110-5c7ab070b7c3 admin admin] The token cache is being invalidate because project 1ba1db4eb9774440b0acc477e68a81c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81732) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:37:23.876036 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-6fcce347-3589-47df-a110-5c7ab070b7c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:37:23.876693 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-6fcce347-3589-47df-a110-5c7ab070b7c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:37:23.878707 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1228/2481] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 24 17:37:23 2022] DELETE /identity/v3/projects/1ba1db4eb9774440b0acc477e68a81c2 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:37:33.240922 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=17 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:37:33.872652 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=70,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:37:53.166204 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b83f79-5bb1-429d-b2c8-01574ee53eb5 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:53.167292 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b83f79-5bb1-429d-b2c8-01574ee53eb5 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:53.167791 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b83f79-5bb1-429d-b2c8-01574ee53eb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:53.190220 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-67b83f79-5bb1-429d-b2c8-01574ee53eb5 None None] MFA Rules not processed for user `0ccc6723f7514e52863d102de3839951`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:37:53.192040 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-67b83f79-5bb1-429d-b2c8-01574ee53eb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:37:53.262565 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1254/2482] 10.222.0.18 () {58 vars in 967 bytes} [Wed Aug 24 17:37:53 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:37:53.286532 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcce347-3589-47df-a110-5c7ab070b7c3 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:37:53.288168 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-6fcce347-3589-47df-a110-5c7ab070b7c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:37:53.369360 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcce347-3589-47df-a110-5c7ab070b7c3 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:37:53.377940 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0241ad2c-889d-4b7a-9a00-6bed90e3679e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef8caa353be34499a58cd25b72ee4032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:37:53.379054 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0241ad2c-889d-4b7a-9a00-6bed90e3679e service swift] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:37:53.379537 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0241ad2c-889d-4b7a-9a00-6bed90e3679e service swift] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:37:53.380017 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0241ad2c-889d-4b7a-9a00-6bed90e3679e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:37:53.395250 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0241ad2c-889d-4b7a-9a00-6bed90e3679e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:37:53.398623 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0241ad2c-889d-4b7a-9a00-6bed90e3679e service swift] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:37:53.413619 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1229/2483] 10.222.0.18 () {60 vars in 1384 bytes} [Wed Aug 24 17:37:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 17:38:03.254833 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=25 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:38:03.407470 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=32 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:38:26.345012 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b83f79-5bb1-429d-b2c8-01574ee53eb5 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:38:26.346550 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-67b83f79-5bb1-429d-b2c8-01574ee53eb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:38:26.423496 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b83f79-5bb1-429d-b2c8-01574ee53eb5 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:38:26.427342 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b158a8-c36f-4e59-8321-069b575a611e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e9ed4d47aef4140bfb2d3bb798cc559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:38:26.427964 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b158a8-c36f-4e59-8321-069b575a611e service neutron] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:26.428267 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b158a8-c36f-4e59-8321-069b575a611e service neutron] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:26.428537 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b158a8-c36f-4e59-8321-069b575a611e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:26.430882 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-e5b158a8-c36f-4e59-8321-069b575a611e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:38:26.513466 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b158a8-c36f-4e59-8321-069b575a611e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:38:26.514824 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b158a8-c36f-4e59-8321-069b575a611e service neutron] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:38:26.525649 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1255/2484] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:38:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:38:30.378250 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0241ad2c-889d-4b7a-9a00-6bed90e3679e service swift] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:38:30.379898 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-0241ad2c-889d-4b7a-9a00-6bed90e3679e service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:38:30.396079 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0241ad2c-889d-4b7a-9a00-6bed90e3679e service swift] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:38:30.400795 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70919fa-2066-45e5-999c-ccff54f8bc7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:38:30.401434 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70919fa-2066-45e5-999c-ccff54f8bc7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:30.401713 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70919fa-2066-45e5-999c-ccff54f8bc7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:30.401985 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70919fa-2066-45e5-999c-ccff54f8bc7f admin admin] PATH_INFO: `/v3/users/ef9e96b709b345c08a4ea10f14211668` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:30.403131 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70919fa-2066-45e5-999c-ccff54f8bc7f admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef9e96b709b345c08a4ea10f14211668)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:38:30.405421 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70919fa-2066-45e5-999c-ccff54f8bc7f admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:38:30.626009 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-e70919fa-2066-45e5-999c-ccff54f8bc7f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef9e96b709b345c08a4ea10f14211668'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:38:30.701914 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-e70919fa-2066-45e5-999c-ccff54f8bc7f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef9e96b709b345c08a4ea10f14211668'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:38:30.702578 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-e70919fa-2066-45e5-999c-ccff54f8bc7f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef9e96b709b345c08a4ea10f14211668'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:38:30.712037 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-e70919fa-2066-45e5-999c-ccff54f8bc7f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef9e96b709b345c08a4ea10f14211668'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:38:30.712794 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-e70919fa-2066-45e5-999c-ccff54f8bc7f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef9e96b709b345c08a4ea10f14211668'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:38:30.730574 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1230/2485] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 24 17:38:30 2022] DELETE /identity/v3/users/ef9e96b709b345c08a4ea10f14211668 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:38:30.983600 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b158a8-c36f-4e59-8321-069b575a611e service neutron] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:38:30.994840 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b158a8-c36f-4e59-8321-069b575a611e service neutron] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:38:31.056413 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ee1a9a-7f78-456b-9802-f89d40827973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:38:31.057045 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ee1a9a-7f78-456b-9802-f89d40827973 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:31.057367 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ee1a9a-7f78-456b-9802-f89d40827973 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:31.057634 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ee1a9a-7f78-456b-9802-f89d40827973 admin admin] PATH_INFO: `/v3/projects/6c8612c530e1495f8d6f4927560a9012` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:31.058731 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ee1a9a-7f78-456b-9802-f89d40827973 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c8612c530e1495f8d6f4927560a9012)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:38:31.060671 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ee1a9a-7f78-456b-9802-f89d40827973 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:38:31.171157 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-c3ee1a9a-7f78-456b-9802-f89d40827973 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c8612c530e1495f8d6f4927560a9012', 'request_id': 'req-c3ee1a9a-7f78-456b-9802-f89d40827973'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:38:31.173778 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:38:31.319119 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-c3ee1a9a-7f78-456b-9802-f89d40827973 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c8612c530e1495f8d6f4927560a9012', 'request_id': 'req-c3ee1a9a-7f78-456b-9802-f89d40827973'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:38:31.326115 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-c3ee1a9a-7f78-456b-9802-f89d40827973 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c8612c530e1495f8d6f4927560a9012', 'request_id': 'req-c3ee1a9a-7f78-456b-9802-f89d40827973'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:38:31.337018 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-c3ee1a9a-7f78-456b-9802-f89d40827973 admin admin] The token cache is being invalidate because project 6c8612c530e1495f8d6f4927560a9012 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81731) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:38:31.337306 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-c3ee1a9a-7f78-456b-9802-f89d40827973 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:38:31.337941 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-c3ee1a9a-7f78-456b-9802-f89d40827973 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:38:31.339493 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1256/2486] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 24 17:38:30 2022] DELETE /identity/v3/projects/6c8612c530e1495f8d6f4927560a9012 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:38:31.344532 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e4873b-2a21-42aa-966a-a0fbb0c9b2f9 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:31.344822 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e4873b-2a21-42aa-966a-a0fbb0c9b2f9 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:31.345098 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e4873b-2a21-42aa-966a-a0fbb0c9b2f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:31.364922 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-13e4873b-2a21-42aa-966a-a0fbb0c9b2f9 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:38:31.372044 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-13e4873b-2a21-42aa-966a-a0fbb0c9b2f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:38:31.428572 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1231/2487] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:38:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:38:31.437528 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79b480f-cabb-4d7a-ac14-9a0d35ba3265 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:31.437821 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79b480f-cabb-4d7a-ac14-9a0d35ba3265 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:31.438101 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79b480f-cabb-4d7a-ac14-9a0d35ba3265 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:31.458129 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-d79b480f-cabb-4d7a-ac14-9a0d35ba3265 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:38:31.459426 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-d79b480f-cabb-4d7a-ac14-9a0d35ba3265 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:38:31.468150 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1257/2488] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:38:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:38:31.472745 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e4873b-2a21-42aa-966a-a0fbb0c9b2f9 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:38:31.482804 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e4873b-2a21-42aa-966a-a0fbb0c9b2f9 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:38:31.487915 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e645ae76-2b25-44f6-bad5-9b8af702a645 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:38:31.488551 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e645ae76-2b25-44f6-bad5-9b8af702a645 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:31.488831 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e645ae76-2b25-44f6-bad5-9b8af702a645 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:31.489102 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e645ae76-2b25-44f6-bad5-9b8af702a645 admin admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:31.489775 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e645ae76-2b25-44f6-bad5-9b8af702a645 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:38:31.490080 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e645ae76-2b25-44f6-bad5-9b8af702a645 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:38:31.491866 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e645ae76-2b25-44f6-bad5-9b8af702a645 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:38:31.503606 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1232/2489] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 24 17:38:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:38:31.508949 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79b480f-cabb-4d7a-ac14-9a0d35ba3265 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:38:31.518413 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79b480f-cabb-4d7a-ac14-9a0d35ba3265 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:38:31.524304 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ef35d3-1471-4d09-9b49-b31a27397193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:38:31.524912 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ef35d3-1471-4d09-9b49-b31a27397193 admin admin] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:31.525189 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ef35d3-1471-4d09-9b49-b31a27397193 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:31.525459 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ef35d3-1471-4d09-9b49-b31a27397193 admin admin] PATH_INFO: `/v3/projects` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:31.526081 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ef35d3-1471-4d09-9b49-b31a27397193 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:38:31.528005 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ef35d3-1471-4d09-9b49-b31a27397193 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:38:31.612576 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1258/2490] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 24 17:38:31 2022] POST /identity/v3/projects => generated 365 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:38:31.617355 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e645ae76-2b25-44f6-bad5-9b8af702a645 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:38:31.627636 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e645ae76-2b25-44f6-bad5-9b8af702a645 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:38:31.684921 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9617ad1c-438e-4d8d-9fec-df5ed4853d06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:38:31.685578 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9617ad1c-438e-4d8d-9fec-df5ed4853d06 admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:31.685863 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9617ad1c-438e-4d8d-9fec-df5ed4853d06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:31.686138 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9617ad1c-438e-4d8d-9fec-df5ed4853d06 admin admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:31.686789 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9617ad1c-438e-4d8d-9fec-df5ed4853d06 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:38:31.688914 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9617ad1c-438e-4d8d-9fec-df5ed4853d06 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:38:31.697131 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:38:31.822172 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1233/2491] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 24 17:38:31 2022] POST /identity/v3/users => generated 283 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:38:31.826404 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ef35d3-1471-4d09-9b49-b31a27397193 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:38:31.837132 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ef35d3-1471-4d09-9b49-b31a27397193 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:38:31.841600 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e679935c-593d-49dc-a1b2-25f5aa32c518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:38:31.842263 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e679935c-593d-49dc-a1b2-25f5aa32c518 admin admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:31.842476 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e679935c-593d-49dc-a1b2-25f5aa32c518 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:31.842766 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e679935c-593d-49dc-a1b2-25f5aa32c518 admin admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:31.843489 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e679935c-593d-49dc-a1b2-25f5aa32c518 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:38:31.845067 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e679935c-593d-49dc-a1b2-25f5aa32c518 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:38:31.855568 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1259/2492] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:38:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:38:31.859726 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9617ad1c-438e-4d8d-9fec-df5ed4853d06 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:38:31.870421 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9617ad1c-438e-4d8d-9fec-df5ed4853d06 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:38:31.875130 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a881850f-d43b-4fae-b1b7-00af5f4b870c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:38:31.875729 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a881850f-d43b-4fae-b1b7-00af5f4b870c admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:31.876014 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a881850f-d43b-4fae-b1b7-00af5f4b870c admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:31.876271 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a881850f-d43b-4fae-b1b7-00af5f4b870c admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:31.876910 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a881850f-d43b-4fae-b1b7-00af5f4b870c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:38:31.878491 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a881850f-d43b-4fae-b1b7-00af5f4b870c admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:38:31.890332 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1234/2493] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:38:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:38:31.894465 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e679935c-593d-49dc-a1b2-25f5aa32c518 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:38:31.906111 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e679935c-593d-49dc-a1b2-25f5aa32c518 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:38:31.910812 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115f8e63-a2fa-4eee-8d7a-fd8a6f168df9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:38:31.911492 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115f8e63-a2fa-4eee-8d7a-fd8a6f168df9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:31.911840 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115f8e63-a2fa-4eee-8d7a-fd8a6f168df9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:31.912135 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115f8e63-a2fa-4eee-8d7a-fd8a6f168df9 admin admin] PATH_INFO: `/v3/projects/78f150844a9e4295bcb846540d4c25da/users/cdb2c96469764e3c9e98b1c94dce78e7/roles/60dc7e16aee746d5a3e8aa27d7d8525c` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:31.927976 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115f8e63-a2fa-4eee-8d7a-fd8a6f168df9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=78f150844a9e4295bcb846540d4c25da, user_id=cdb2c96469764e3c9e98b1c94dce78e7, role_id=60dc7e16aee746d5a3e8aa27d7d8525c)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:38:31.931433 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115f8e63-a2fa-4eee-8d7a-fd8a6f168df9 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:38:32.016911 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1260/2494] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 24 17:38:31 2022] PUT /identity/v3/projects/78f150844a9e4295bcb846540d4c25da/users/cdb2c96469764e3c9e98b1c94dce78e7/roles/60dc7e16aee746d5a3e8aa27d7d8525c => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:38:32.027845 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27e367e-aff1-4856-8031-f5ef4e4bc2a1 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:32.028146 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27e367e-aff1-4856-8031-f5ef4e4bc2a1 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:32.028389 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27e367e-aff1-4856-8031-f5ef4e4bc2a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:32.056689 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-e27e367e-aff1-4856-8031-f5ef4e4bc2a1 None None] MFA Rules not processed for user `cdb2c96469764e3c9e98b1c94dce78e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:38:32.057954 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-e27e367e-aff1-4856-8031-f5ef4e4bc2a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:38:32.122939 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1235/2495] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:38:32 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:38:32.127395 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115f8e63-a2fa-4eee-8d7a-fd8a6f168df9 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:38:32.192131 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115f8e63-a2fa-4eee-8d7a-fd8a6f168df9 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:38:32.197058 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0937ea4b-f9b8-46bf-8d5e-12f8182b5b6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:38:32.197732 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0937ea4b-f9b8-46bf-8d5e-12f8182b5b6c admin admin] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:32.198010 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0937ea4b-f9b8-46bf-8d5e-12f8182b5b6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:32.198286 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0937ea4b-f9b8-46bf-8d5e-12f8182b5b6c admin admin] PATH_INFO: `/v3/projects` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:32.198894 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0937ea4b-f9b8-46bf-8d5e-12f8182b5b6c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:38:32.200835 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0937ea4b-f9b8-46bf-8d5e-12f8182b5b6c admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:38:32.286222 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1261/2496] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 24 17:38:32 2022] POST /identity/v3/projects => generated 365 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:38:32.291010 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27e367e-aff1-4856-8031-f5ef4e4bc2a1 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:38:32.301798 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27e367e-aff1-4856-8031-f5ef4e4bc2a1 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:38:32.362682 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb86b17-7b29-4a1d-8109-bf77b67e20c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:38:32.363326 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb86b17-7b29-4a1d-8109-bf77b67e20c3 admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:32.363610 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb86b17-7b29-4a1d-8109-bf77b67e20c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:32.363881 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb86b17-7b29-4a1d-8109-bf77b67e20c3 admin admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:32.364504 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb86b17-7b29-4a1d-8109-bf77b67e20c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:38:32.366452 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb86b17-7b29-4a1d-8109-bf77b67e20c3 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:38:32.467730 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1236/2497] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 24 17:38:32 2022] POST /identity/v3/users => generated 289 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:38:32.471827 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0937ea4b-f9b8-46bf-8d5e-12f8182b5b6c admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:38:32.482967 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0937ea4b-f9b8-46bf-8d5e-12f8182b5b6c admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:38:32.487662 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95798477-ed4c-4bfc-b225-193efd90e671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:38:32.488263 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95798477-ed4c-4bfc-b225-193efd90e671 admin admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:32.488583 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95798477-ed4c-4bfc-b225-193efd90e671 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:32.488913 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95798477-ed4c-4bfc-b225-193efd90e671 admin admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:32.489554 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95798477-ed4c-4bfc-b225-193efd90e671 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:38:32.491176 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95798477-ed4c-4bfc-b225-193efd90e671 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:38:32.502452 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1262/2498] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:38:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:38:32.506712 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb86b17-7b29-4a1d-8109-bf77b67e20c3 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:38:32.516739 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb86b17-7b29-4a1d-8109-bf77b67e20c3 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:38:32.521613 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff40aa45-13aa-4101-9d37-ac6be6f75066 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:38:32.522264 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff40aa45-13aa-4101-9d37-ac6be6f75066 admin admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:32.522545 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff40aa45-13aa-4101-9d37-ac6be6f75066 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:32.522822 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff40aa45-13aa-4101-9d37-ac6be6f75066 admin admin] PATH_INFO: `/v3/projects/11d738e0124e4a578b19be1093a1e216/users/b612df26172547118afab0b2e29b44a0/roles/c5346bc54abf445f9900edf94264c850` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:32.544591 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff40aa45-13aa-4101-9d37-ac6be6f75066 admin admin] RBAC: Authorizing `identity:create_grant(project_id=11d738e0124e4a578b19be1093a1e216, user_id=b612df26172547118afab0b2e29b44a0, role_id=c5346bc54abf445f9900edf94264c850)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:38:32.547641 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff40aa45-13aa-4101-9d37-ac6be6f75066 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:38:32.633511 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1237/2499] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 24 17:38:32 2022] PUT /identity/v3/projects/11d738e0124e4a578b19be1093a1e216/users/b612df26172547118afab0b2e29b44a0/roles/c5346bc54abf445f9900edf94264c850 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:38:32.637639 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95798477-ed4c-4bfc-b225-193efd90e671 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:38:32.648131 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95798477-ed4c-4bfc-b225-193efd90e671 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:38:32.697315 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:38:32.707185 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55727bdc-52e0-4cab-9076-cd1facd15589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:38:32.709516 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55727bdc-52e0-4cab-9076-cd1facd15589 admin admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:32.709516 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55727bdc-52e0-4cab-9076-cd1facd15589 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:32.709516 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55727bdc-52e0-4cab-9076-cd1facd15589 admin admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:32.709516 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55727bdc-52e0-4cab-9076-cd1facd15589 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:38:32.711622 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55727bdc-52e0-4cab-9076-cd1facd15589 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:38:32.721073 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1263/2500] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:38:32 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:38:32.725325 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff40aa45-13aa-4101-9d37-ac6be6f75066 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:38:32.735991 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff40aa45-13aa-4101-9d37-ac6be6f75066 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:38:32.740729 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a410dbc7-6ebf-45ef-8702-f4b529bc428a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:38:32.741397 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a410dbc7-6ebf-45ef-8702-f4b529bc428a admin admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:32.742204 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a410dbc7-6ebf-45ef-8702-f4b529bc428a admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:32.742478 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a410dbc7-6ebf-45ef-8702-f4b529bc428a admin admin] PATH_INFO: `/v3/projects/11d738e0124e4a578b19be1093a1e216/users/b612df26172547118afab0b2e29b44a0/roles/c5346bc54abf445f9900edf94264c850` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:32.744571 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a410dbc7-6ebf-45ef-8702-f4b529bc428a admin admin] RBAC: Authorizing `identity:create_grant(project_id=11d738e0124e4a578b19be1093a1e216, user_id=b612df26172547118afab0b2e29b44a0, role_id=c5346bc54abf445f9900edf94264c850)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:38:32.747469 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a410dbc7-6ebf-45ef-8702-f4b529bc428a admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:38:32.758851 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1238/2501] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 24 17:38:32 2022] PUT /identity/v3/projects/11d738e0124e4a578b19be1093a1e216/users/b612df26172547118afab0b2e29b44a0/roles/c5346bc54abf445f9900edf94264c850 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:38:32.771705 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990afc13-e375-468f-94d6-773b506a7cb7 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:32.772054 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990afc13-e375-468f-94d6-773b506a7cb7 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:32.772554 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990afc13-e375-468f-94d6-773b506a7cb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:32.800857 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-990afc13-e375-468f-94d6-773b506a7cb7 None None] MFA Rules not processed for user `b612df26172547118afab0b2e29b44a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:38:32.802127 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-990afc13-e375-468f-94d6-773b506a7cb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:38:32.872022 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1264/2502] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:38:32 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:38:32.884067 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a410dbc7-6ebf-45ef-8702-f4b529bc428a admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:38:32.885886 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-a410dbc7-6ebf-45ef-8702-f4b529bc428a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:38:32.909368 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:38:32.947509 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a410dbc7-6ebf-45ef-8702-f4b529bc428a admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:38:32.951151 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd0ea72-7945-4dbd-89f7-19413209924d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be06246d47584b529150ceb98a5cfd95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:38:32.951757 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd0ea72-7945-4dbd-89f7-19413209924d service cinder] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:32.952120 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd0ea72-7945-4dbd-89f7-19413209924d service cinder] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:32.952418 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd0ea72-7945-4dbd-89f7-19413209924d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:32.961978 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd0ea72-7945-4dbd-89f7-19413209924d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:38:32.963588 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd0ea72-7945-4dbd-89f7-19413209924d service cinder] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:38:32.974454 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1239/2503] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:38:32 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:38:33.283019 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990afc13-e375-468f-94d6-773b506a7cb7 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:38:33.293251 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990afc13-e375-468f-94d6-773b506a7cb7 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:38:33.297092 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e58ec0-10dd-4b19-80a2-717bb46f29e2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be06246d47584b529150ceb98a5cfd95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:38:33.297773 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e58ec0-10dd-4b19-80a2-717bb46f29e2 service cinder] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:33.298121 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e58ec0-10dd-4b19-80a2-717bb46f29e2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:33.298395 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e58ec0-10dd-4b19-80a2-717bb46f29e2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:33.356589 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e58ec0-10dd-4b19-80a2-717bb46f29e2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:38:33.358049 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e58ec0-10dd-4b19-80a2-717bb46f29e2 service cinder] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:38:33.367831 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1265/2504] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:38:33 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:38:42.969808 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=17 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:38:43.362978 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=68 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:38:48.620692 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd0ea72-7945-4dbd-89f7-19413209924d service cinder] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:38:48.693043 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd0ea72-7945-4dbd-89f7-19413209924d service cinder] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:38:48.698365 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd851709-da51-4035-8ce8-038e8ffff5a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:38:48.699013 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd851709-da51-4035-8ce8-038e8ffff5a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:48.699294 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd851709-da51-4035-8ce8-038e8ffff5a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:48.699566 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd851709-da51-4035-8ce8-038e8ffff5a3 admin admin] PATH_INFO: `/v3/users/cdb2c96469764e3c9e98b1c94dce78e7` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:48.700751 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd851709-da51-4035-8ce8-038e8ffff5a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cdb2c96469764e3c9e98b1c94dce78e7)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:38:48.702648 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd851709-da51-4035-8ce8-038e8ffff5a3 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:38:48.932905 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-dd851709-da51-4035-8ce8-038e8ffff5a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cdb2c96469764e3c9e98b1c94dce78e7'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:38:49.010382 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-dd851709-da51-4035-8ce8-038e8ffff5a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cdb2c96469764e3c9e98b1c94dce78e7'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:38:49.011178 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-dd851709-da51-4035-8ce8-038e8ffff5a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cdb2c96469764e3c9e98b1c94dce78e7'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:38:49.018789 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-dd851709-da51-4035-8ce8-038e8ffff5a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cdb2c96469764e3c9e98b1c94dce78e7'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:38:49.019492 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-dd851709-da51-4035-8ce8-038e8ffff5a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cdb2c96469764e3c9e98b1c94dce78e7'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:38:49.038063 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1240/2505] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 24 17:38:48 2022] DELETE /identity/v3/users/cdb2c96469764e3c9e98b1c94dce78e7 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:38:49.042621 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb9294c-3391-4790-b1d8-864f531cc17a None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:49.043167 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb9294c-3391-4790-b1d8-864f531cc17a None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:49.043294 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb9294c-3391-4790-b1d8-864f531cc17a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:49.063154 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-cfb9294c-3391-4790-b1d8-864f531cc17a None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:38:49.064508 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-cfb9294c-3391-4790-b1d8-864f531cc17a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:38:49.125555 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1266/2506] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:38:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:38:49.135759 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd851709-da51-4035-8ce8-038e8ffff5a3 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:38:49.137228 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-dd851709-da51-4035-8ce8-038e8ffff5a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:38:49.197061 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd851709-da51-4035-8ce8-038e8ffff5a3 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:38:49.200638 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b388e4-36eb-470d-b32c-511959370eea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e9ed4d47aef4140bfb2d3bb798cc559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:38:49.201334 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b388e4-36eb-470d-b32c-511959370eea service neutron] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:49.201590 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b388e4-36eb-470d-b32c-511959370eea service neutron] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:49.201864 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b388e4-36eb-470d-b32c-511959370eea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:49.211079 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b388e4-36eb-470d-b32c-511959370eea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:38:49.218133 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b388e4-36eb-470d-b32c-511959370eea service neutron] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:38:49.222798 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1241/2507] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:38:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:38:49.635022 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb9294c-3391-4790-b1d8-864f531cc17a None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:38:49.636464 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-cfb9294c-3391-4790-b1d8-864f531cc17a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:38:49.648933 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb9294c-3391-4790-b1d8-864f531cc17a None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:38:49.655141 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880ece81-23a6-4740-8b00-3ee6e743a335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:38:49.655760 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880ece81-23a6-4740-8b00-3ee6e743a335 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:49.656046 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880ece81-23a6-4740-8b00-3ee6e743a335 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:49.656314 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880ece81-23a6-4740-8b00-3ee6e743a335 admin admin] PATH_INFO: `/v3/projects/78f150844a9e4295bcb846540d4c25da` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:49.657496 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880ece81-23a6-4740-8b00-3ee6e743a335 admin admin] RBAC: Authorizing `identity:delete_project(project_id=78f150844a9e4295bcb846540d4c25da)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:38:49.659461 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880ece81-23a6-4740-8b00-3ee6e743a335 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:38:49.816509 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-880ece81-23a6-4740-8b00-3ee6e743a335 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78f150844a9e4295bcb846540d4c25da', 'request_id': 'req-880ece81-23a6-4740-8b00-3ee6e743a335'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:38:49.901814 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-880ece81-23a6-4740-8b00-3ee6e743a335 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78f150844a9e4295bcb846540d4c25da', 'request_id': 'req-880ece81-23a6-4740-8b00-3ee6e743a335'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:38:49.908747 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-880ece81-23a6-4740-8b00-3ee6e743a335 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78f150844a9e4295bcb846540d4c25da', 'request_id': 'req-880ece81-23a6-4740-8b00-3ee6e743a335'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:38:49.919545 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-880ece81-23a6-4740-8b00-3ee6e743a335 admin admin] The token cache is being invalidate because project 78f150844a9e4295bcb846540d4c25da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81731) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:38:49.919830 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-880ece81-23a6-4740-8b00-3ee6e743a335 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:38:49.920480 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-880ece81-23a6-4740-8b00-3ee6e743a335 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:38:49.921962 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1267/2508] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 24 17:38:49 2022] DELETE /identity/v3/projects/78f150844a9e4295bcb846540d4c25da => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:38:49.926092 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b388e4-36eb-470d-b32c-511959370eea service neutron] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:38:49.935949 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b388e4-36eb-470d-b32c-511959370eea service neutron] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:38:49.954677 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:38:49.999709 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375040ab-3c4d-4e35-9f7a-34d9cdaa5463 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:38:50.000334 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375040ab-3c4d-4e35-9f7a-34d9cdaa5463 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:50.000609 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375040ab-3c4d-4e35-9f7a-34d9cdaa5463 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:50.000877 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375040ab-3c4d-4e35-9f7a-34d9cdaa5463 admin admin] PATH_INFO: `/v3/users/b612df26172547118afab0b2e29b44a0` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:50.001993 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375040ab-3c4d-4e35-9f7a-34d9cdaa5463 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b612df26172547118afab0b2e29b44a0)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:38:50.003965 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375040ab-3c4d-4e35-9f7a-34d9cdaa5463 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:38:50.259019 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-375040ab-3c4d-4e35-9f7a-34d9cdaa5463 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b612df26172547118afab0b2e29b44a0'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:38:50.330384 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-375040ab-3c4d-4e35-9f7a-34d9cdaa5463 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b612df26172547118afab0b2e29b44a0'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:38:50.331111 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-375040ab-3c4d-4e35-9f7a-34d9cdaa5463 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b612df26172547118afab0b2e29b44a0'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:38:50.339102 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-375040ab-3c4d-4e35-9f7a-34d9cdaa5463 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b612df26172547118afab0b2e29b44a0'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:38:50.339869 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-375040ab-3c4d-4e35-9f7a-34d9cdaa5463 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b612df26172547118afab0b2e29b44a0'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:38:50.358205 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1242/2509] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 24 17:38:49 2022] DELETE /identity/v3/users/b612df26172547118afab0b2e29b44a0 => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:38:50.784113 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880ece81-23a6-4740-8b00-3ee6e743a335 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:38:50.794234 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-880ece81-23a6-4740-8b00-3ee6e743a335 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:38:50.800633 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880ece81-23a6-4740-8b00-3ee6e743a335 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:38:50.859823 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29ffda9-9cef-49d4-9a0e-f1c663d4cb0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:38:50.860459 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29ffda9-9cef-49d4-9a0e-f1c663d4cb0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:50.860736 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29ffda9-9cef-49d4-9a0e-f1c663d4cb0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:50.861003 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29ffda9-9cef-49d4-9a0e-f1c663d4cb0f admin admin] PATH_INFO: `/v3/projects/11d738e0124e4a578b19be1093a1e216` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:50.862168 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29ffda9-9cef-49d4-9a0e-f1c663d4cb0f admin admin] RBAC: Authorizing `identity:delete_project(project_id=11d738e0124e4a578b19be1093a1e216)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:38:50.864085 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29ffda9-9cef-49d4-9a0e-f1c663d4cb0f admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:38:50.937494 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:38:50.953773 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-c29ffda9-9cef-49d4-9a0e-f1c663d4cb0f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '11d738e0124e4a578b19be1093a1e216', 'request_id': 'req-c29ffda9-9cef-49d4-9a0e-f1c663d4cb0f'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:38:51.003930 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-c29ffda9-9cef-49d4-9a0e-f1c663d4cb0f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '11d738e0124e4a578b19be1093a1e216', 'request_id': 'req-c29ffda9-9cef-49d4-9a0e-f1c663d4cb0f'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:38:51.010717 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-c29ffda9-9cef-49d4-9a0e-f1c663d4cb0f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '11d738e0124e4a578b19be1093a1e216', 'request_id': 'req-c29ffda9-9cef-49d4-9a0e-f1c663d4cb0f'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:38:51.022669 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-c29ffda9-9cef-49d4-9a0e-f1c663d4cb0f admin admin] The token cache is being invalidate because project 11d738e0124e4a578b19be1093a1e216 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81731) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:38:51.022996 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-c29ffda9-9cef-49d4-9a0e-f1c663d4cb0f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:38:51.023651 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-c29ffda9-9cef-49d4-9a0e-f1c663d4cb0f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:38:51.025171 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1268/2510] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 24 17:38:50 2022] DELETE /identity/v3/projects/11d738e0124e4a578b19be1093a1e216 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:38:51.030424 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331f1897-ce22-4866-b89b-df1b50f8da54 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:51.031021 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331f1897-ce22-4866-b89b-df1b50f8da54 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:51.031130 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331f1897-ce22-4866-b89b-df1b50f8da54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:51.053195 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-331f1897-ce22-4866-b89b-df1b50f8da54 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:38:51.053913 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-331f1897-ce22-4866-b89b-df1b50f8da54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:38:51.115131 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1243/2511] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:38:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:38:51.123984 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194dc27a-0370-4d70-a6d1-2ee356b12ca5 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:51.124300 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194dc27a-0370-4d70-a6d1-2ee356b12ca5 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:51.124578 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194dc27a-0370-4d70-a6d1-2ee356b12ca5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:51.145250 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-194dc27a-0370-4d70-a6d1-2ee356b12ca5 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:38:51.146517 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-194dc27a-0370-4d70-a6d1-2ee356b12ca5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:38:51.154980 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1269/2512] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:38:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:38:51.159344 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331f1897-ce22-4866-b89b-df1b50f8da54 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:38:51.170040 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331f1897-ce22-4866-b89b-df1b50f8da54 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:38:51.174816 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe183e8-25b9-4711-990f-37a7d79915da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:38:51.175591 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe183e8-25b9-4711-990f-37a7d79915da admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:51.175750 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe183e8-25b9-4711-990f-37a7d79915da admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:51.176013 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe183e8-25b9-4711-990f-37a7d79915da admin admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:51.176675 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe183e8-25b9-4711-990f-37a7d79915da admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:38:51.176972 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe183e8-25b9-4711-990f-37a7d79915da admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:38:51.178570 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe183e8-25b9-4711-990f-37a7d79915da admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:38:51.192417 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1244/2513] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 24 17:38:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:38:51.197043 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194dc27a-0370-4d70-a6d1-2ee356b12ca5 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:38:51.206800 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194dc27a-0370-4d70-a6d1-2ee356b12ca5 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:38:51.211450 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2692e2-3f5c-4012-9aa9-52d5e35b2076 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:38:51.212045 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2692e2-3f5c-4012-9aa9-52d5e35b2076 admin admin] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:51.212321 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2692e2-3f5c-4012-9aa9-52d5e35b2076 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:51.212589 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2692e2-3f5c-4012-9aa9-52d5e35b2076 admin admin] PATH_INFO: `/v3/projects` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:51.213226 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2692e2-3f5c-4012-9aa9-52d5e35b2076 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:38:51.215176 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2692e2-3f5c-4012-9aa9-52d5e35b2076 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:38:51.281005 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1270/2514] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 24 17:38:51 2022] POST /identity/v3/projects => generated 367 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:38:51.285466 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe183e8-25b9-4711-990f-37a7d79915da admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:38:51.295646 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe183e8-25b9-4711-990f-37a7d79915da admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:38:51.322383 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:38:51.354075 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e4fe48-75bb-4da0-a018-577ae19f6887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:38:51.354712 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e4fe48-75bb-4da0-a018-577ae19f6887 admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:51.355039 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e4fe48-75bb-4da0-a018-577ae19f6887 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:51.355309 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e4fe48-75bb-4da0-a018-577ae19f6887 admin admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:51.355957 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e4fe48-75bb-4da0-a018-577ae19f6887 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:38:51.357927 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e4fe48-75bb-4da0-a018-577ae19f6887 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:38:51.419163 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1245/2515] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 24 17:38:51 2022] POST /identity/v3/users => generated 284 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:38:51.420320 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2692e2-3f5c-4012-9aa9-52d5e35b2076 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:38:51.430869 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2692e2-3f5c-4012-9aa9-52d5e35b2076 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:38:51.436181 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee051217-0027-49f8-a56c-c883e13a8f08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:38:51.436782 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee051217-0027-49f8-a56c-c883e13a8f08 admin admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:51.437137 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee051217-0027-49f8-a56c-c883e13a8f08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:51.437416 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee051217-0027-49f8-a56c-c883e13a8f08 admin admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:51.438075 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee051217-0027-49f8-a56c-c883e13a8f08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:38:51.439718 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee051217-0027-49f8-a56c-c883e13a8f08 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:38:51.450656 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1271/2516] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:38:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:38:51.454686 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e4fe48-75bb-4da0-a018-577ae19f6887 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:38:51.464775 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e4fe48-75bb-4da0-a018-577ae19f6887 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:38:51.471027 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d9c907-eb81-4af9-9a09-06e17ebfc262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:38:51.471027 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d9c907-eb81-4af9-9a09-06e17ebfc262 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:51.471360 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d9c907-eb81-4af9-9a09-06e17ebfc262 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:51.471624 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d9c907-eb81-4af9-9a09-06e17ebfc262 admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:51.472269 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d9c907-eb81-4af9-9a09-06e17ebfc262 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:38:51.473899 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d9c907-eb81-4af9-9a09-06e17ebfc262 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:38:51.485425 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1246/2517] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:38:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:38:51.489809 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee051217-0027-49f8-a56c-c883e13a8f08 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:38:51.503366 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee051217-0027-49f8-a56c-c883e13a8f08 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:38:51.508067 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962a3d27-661e-47cc-96e4-8ebc570a4c92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:38:51.509394 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962a3d27-661e-47cc-96e4-8ebc570a4c92 admin admin] REQUEST_METHOD: `PUT` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:51.509673 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962a3d27-661e-47cc-96e4-8ebc570a4c92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:51.509949 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962a3d27-661e-47cc-96e4-8ebc570a4c92 admin admin] PATH_INFO: `/v3/projects/0b811eaeb5434bed909564449b947061/users/59ff43e8e66448e28182877c407868db/roles/60dc7e16aee746d5a3e8aa27d7d8525c` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:51.525530 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962a3d27-661e-47cc-96e4-8ebc570a4c92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b811eaeb5434bed909564449b947061, user_id=59ff43e8e66448e28182877c407868db, role_id=60dc7e16aee746d5a3e8aa27d7d8525c)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:38:51.528846 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962a3d27-661e-47cc-96e4-8ebc570a4c92 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:38:51.599304 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1272/2518] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 24 17:38:51 2022] PUT /identity/v3/projects/0b811eaeb5434bed909564449b947061/users/59ff43e8e66448e28182877c407868db/roles/60dc7e16aee746d5a3e8aa27d7d8525c => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:38:51.610227 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eb0474-013b-4aa0-913d-50a367f55c83 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:51.610517 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eb0474-013b-4aa0-913d-50a367f55c83 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:51.610794 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eb0474-013b-4aa0-913d-50a367f55c83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:51.641416 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-65eb0474-013b-4aa0-913d-50a367f55c83 None None] MFA Rules not processed for user `59ff43e8e66448e28182877c407868db`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:38:51.642660 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-65eb0474-013b-4aa0-913d-50a367f55c83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:38:51.708752 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1247/2519] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:38:51 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:38:51.713265 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962a3d27-661e-47cc-96e4-8ebc570a4c92 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:38:51.774552 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962a3d27-661e-47cc-96e4-8ebc570a4c92 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:38:51.779588 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f208f735-85f0-473b-a83e-ade80b5c983c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:38:51.780186 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f208f735-85f0-473b-a83e-ade80b5c983c admin admin] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:51.780462 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f208f735-85f0-473b-a83e-ade80b5c983c admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:51.780731 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f208f735-85f0-473b-a83e-ade80b5c983c admin admin] PATH_INFO: `/v3/projects` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:51.781344 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f208f735-85f0-473b-a83e-ade80b5c983c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:38:51.783531 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f208f735-85f0-473b-a83e-ade80b5c983c admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:38:51.875199 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1273/2520] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 24 17:38:51 2022] POST /identity/v3/projects => generated 369 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:38:51.879806 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65eb0474-013b-4aa0-913d-50a367f55c83 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:38:51.890671 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65eb0474-013b-4aa0-913d-50a367f55c83 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:38:51.949578 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ca5501-44a5-4dac-ab2e-e66f558bddf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:38:51.950192 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ca5501-44a5-4dac-ab2e-e66f558bddf0 admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:51.950472 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ca5501-44a5-4dac-ab2e-e66f558bddf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:51.950734 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ca5501-44a5-4dac-ab2e-e66f558bddf0 admin admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:51.951384 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ca5501-44a5-4dac-ab2e-e66f558bddf0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:38:51.953312 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ca5501-44a5-4dac-ab2e-e66f558bddf0 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:38:52.045455 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1248/2521] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 24 17:38:51 2022] POST /identity/v3/users => generated 291 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:38:52.049390 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f208f735-85f0-473b-a83e-ade80b5c983c admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:38:52.060575 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f208f735-85f0-473b-a83e-ade80b5c983c admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:38:52.065332 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedf0d3d-6960-45de-9ffd-51ee7af001f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:38:52.065935 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedf0d3d-6960-45de-9ffd-51ee7af001f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:52.066265 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedf0d3d-6960-45de-9ffd-51ee7af001f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:52.066521 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedf0d3d-6960-45de-9ffd-51ee7af001f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:52.067258 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedf0d3d-6960-45de-9ffd-51ee7af001f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:38:52.068947 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedf0d3d-6960-45de-9ffd-51ee7af001f0 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:38:52.080349 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1274/2522] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:38:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:38:52.084551 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ca5501-44a5-4dac-ab2e-e66f558bddf0 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:38:52.095491 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ca5501-44a5-4dac-ab2e-e66f558bddf0 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:38:52.100069 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e601c0-fb4b-49d1-b129-7eda27fab7d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:38:52.100715 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e601c0-fb4b-49d1-b129-7eda27fab7d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:52.100993 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e601c0-fb4b-49d1-b129-7eda27fab7d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:52.101261 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e601c0-fb4b-49d1-b129-7eda27fab7d6 admin admin] PATH_INFO: `/v3/projects/61b1d4d09e7e47deb59f9802ff305971/users/b9c6cfc4f7d14323a6890c73b30ef608/roles/c5346bc54abf445f9900edf94264c850` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:52.116439 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e601c0-fb4b-49d1-b129-7eda27fab7d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=61b1d4d09e7e47deb59f9802ff305971, user_id=b9c6cfc4f7d14323a6890c73b30ef608, role_id=c5346bc54abf445f9900edf94264c850)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:38:52.119169 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e601c0-fb4b-49d1-b129-7eda27fab7d6 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:38:52.233545 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1249/2523] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 24 17:38:52 2022] PUT /identity/v3/projects/61b1d4d09e7e47deb59f9802ff305971/users/b9c6cfc4f7d14323a6890c73b30ef608/roles/c5346bc54abf445f9900edf94264c850 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:38:52.237560 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedf0d3d-6960-45de-9ffd-51ee7af001f0 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:38:52.248464 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedf0d3d-6960-45de-9ffd-51ee7af001f0 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:38:52.269541 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:38:52.305501 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cc7e92-fbea-44eb-bcd4-16d72c764826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:38:52.306109 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cc7e92-fbea-44eb-bcd4-16d72c764826 admin admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:52.306387 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cc7e92-fbea-44eb-bcd4-16d72c764826 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:52.306656 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cc7e92-fbea-44eb-bcd4-16d72c764826 admin admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:52.307325 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cc7e92-fbea-44eb-bcd4-16d72c764826 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:38:52.309599 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cc7e92-fbea-44eb-bcd4-16d72c764826 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:38:52.320229 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1275/2524] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:38:52 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:38:52.324069 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e601c0-fb4b-49d1-b129-7eda27fab7d6 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:38:52.336491 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:38:52.342224 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e601c0-fb4b-49d1-b129-7eda27fab7d6 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:38:52.347121 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9ffc5f-f18c-4961-ae63-456b662af612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:38:52.347792 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9ffc5f-f18c-4961-ae63-456b662af612 admin admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:52.348073 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9ffc5f-f18c-4961-ae63-456b662af612 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:52.348355 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9ffc5f-f18c-4961-ae63-456b662af612 admin admin] PATH_INFO: `/v3/projects/61b1d4d09e7e47deb59f9802ff305971/users/b9c6cfc4f7d14323a6890c73b30ef608/roles/c5346bc54abf445f9900edf94264c850` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:52.350374 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9ffc5f-f18c-4961-ae63-456b662af612 admin admin] RBAC: Authorizing `identity:create_grant(project_id=61b1d4d09e7e47deb59f9802ff305971, user_id=b9c6cfc4f7d14323a6890c73b30ef608, role_id=c5346bc54abf445f9900edf94264c850)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:38:52.353182 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9ffc5f-f18c-4961-ae63-456b662af612 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:38:52.364542 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1250/2525] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 24 17:38:52 2022] PUT /identity/v3/projects/61b1d4d09e7e47deb59f9802ff305971/users/b9c6cfc4f7d14323a6890c73b30ef608/roles/c5346bc54abf445f9900edf94264c850 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:38:52.376613 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c053b0f-484d-48ee-a2ce-2461beff53bd None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:52.376895 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c053b0f-484d-48ee-a2ce-2461beff53bd None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:52.377171 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c053b0f-484d-48ee-a2ce-2461beff53bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:52.405984 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-6c053b0f-484d-48ee-a2ce-2461beff53bd None None] MFA Rules not processed for user `b9c6cfc4f7d14323a6890c73b30ef608`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:38:52.407257 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-6c053b0f-484d-48ee-a2ce-2461beff53bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:38:52.479437 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1276/2526] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:38:52 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:38:52.491538 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9ffc5f-f18c-4961-ae63-456b662af612 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:38:52.493288 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-dc9ffc5f-f18c-4961-ae63-456b662af612 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:38:52.553326 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9ffc5f-f18c-4961-ae63-456b662af612 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:38:52.557026 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42487403-d6b9-4501-8689-e5cf68ece0de service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be06246d47584b529150ceb98a5cfd95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:38:52.557644 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42487403-d6b9-4501-8689-e5cf68ece0de service cinder] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:52.557925 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42487403-d6b9-4501-8689-e5cf68ece0de service cinder] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:52.558199 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42487403-d6b9-4501-8689-e5cf68ece0de service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:52.569719 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42487403-d6b9-4501-8689-e5cf68ece0de service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:38:52.571235 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42487403-d6b9-4501-8689-e5cf68ece0de service cinder] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:38:52.581235 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1251/2527] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:38:52 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:38:52.723362 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c053b0f-484d-48ee-a2ce-2461beff53bd None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:38:52.724747 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-6c053b0f-484d-48ee-a2ce-2461beff53bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:38:52.737667 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c053b0f-484d-48ee-a2ce-2461beff53bd None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:38:52.741188 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f2ec0c-40a8-4976-85ac-07b10498a87a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be06246d47584b529150ceb98a5cfd95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:38:52.741782 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f2ec0c-40a8-4976-85ac-07b10498a87a service cinder] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:52.742064 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f2ec0c-40a8-4976-85ac-07b10498a87a service cinder] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:52.742331 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f2ec0c-40a8-4976-85ac-07b10498a87a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:52.802619 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f2ec0c-40a8-4976-85ac-07b10498a87a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:38:52.804103 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f2ec0c-40a8-4976-85ac-07b10498a87a service cinder] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:38:52.812943 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1277/2528] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:38:52 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:38:57.608872 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42487403-d6b9-4501-8689-e5cf68ece0de service cinder] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:38:57.673907 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42487403-d6b9-4501-8689-e5cf68ece0de service cinder] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:38:57.678616 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9a29c3-c637-4f16-a1ad-15ed7754d07b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:38:57.679276 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9a29c3-c637-4f16-a1ad-15ed7754d07b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:57.679553 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9a29c3-c637-4f16-a1ad-15ed7754d07b admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:57.679823 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9a29c3-c637-4f16-a1ad-15ed7754d07b admin admin] PATH_INFO: `/v3/users/59ff43e8e66448e28182877c407868db` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:57.680983 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9a29c3-c637-4f16-a1ad-15ed7754d07b admin admin] RBAC: Authorizing `identity:delete_user(user_id=59ff43e8e66448e28182877c407868db)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:38:57.683029 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9a29c3-c637-4f16-a1ad-15ed7754d07b admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:38:57.869602 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-3c9a29c3-c637-4f16-a1ad-15ed7754d07b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59ff43e8e66448e28182877c407868db'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:38:57.952857 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-3c9a29c3-c637-4f16-a1ad-15ed7754d07b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59ff43e8e66448e28182877c407868db'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:38:57.953552 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-3c9a29c3-c637-4f16-a1ad-15ed7754d07b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59ff43e8e66448e28182877c407868db'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:38:57.961390 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-3c9a29c3-c637-4f16-a1ad-15ed7754d07b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59ff43e8e66448e28182877c407868db'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:38:57.962030 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-3c9a29c3-c637-4f16-a1ad-15ed7754d07b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59ff43e8e66448e28182877c407868db'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:38:57.980046 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1252/2529] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 24 17:38:57 2022] DELETE /identity/v3/users/59ff43e8e66448e28182877c407868db => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:38:57.984543 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e878396c-6550-43a1-bd02-ce922be89d0b None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:57.985049 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e878396c-6550-43a1-bd02-ce922be89d0b None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:57.985232 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e878396c-6550-43a1-bd02-ce922be89d0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:58.003699 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-e878396c-6550-43a1-bd02-ce922be89d0b None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:38:58.004880 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-e878396c-6550-43a1-bd02-ce922be89d0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:38:58.044710 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:38:58.062817 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1278/2530] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:38:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:38:58.073044 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9a29c3-c637-4f16-a1ad-15ed7754d07b admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:38:58.075017 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-3c9a29c3-c637-4f16-a1ad-15ed7754d07b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:38:58.088773 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:38:58.133635 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9a29c3-c637-4f16-a1ad-15ed7754d07b admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:38:58.138766 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c93377-59ce-400d-a400-b4b710f36125 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e9ed4d47aef4140bfb2d3bb798cc559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:38:58.139421 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c93377-59ce-400d-a400-b4b710f36125 service neutron] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:58.139705 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c93377-59ce-400d-a400-b4b710f36125 service neutron] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:58.140009 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c93377-59ce-400d-a400-b4b710f36125 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:58.149411 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c93377-59ce-400d-a400-b4b710f36125 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:38:58.150842 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c93377-59ce-400d-a400-b4b710f36125 service neutron] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:38:58.161557 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1253/2531] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:38:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:38:58.822950 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e878396c-6550-43a1-bd02-ce922be89d0b None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:38:58.824472 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-e878396c-6550-43a1-bd02-ce922be89d0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:38:58.838800 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e878396c-6550-43a1-bd02-ce922be89d0b None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:38:58.844281 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae5b910-c465-4366-9b00-18e6a8770136 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:38:58.845153 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae5b910-c465-4366-9b00-18e6a8770136 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:58.845516 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae5b910-c465-4366-9b00-18e6a8770136 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:58.845734 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae5b910-c465-4366-9b00-18e6a8770136 admin admin] PATH_INFO: `/v3/projects/0b811eaeb5434bed909564449b947061` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:58.846857 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae5b910-c465-4366-9b00-18e6a8770136 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b811eaeb5434bed909564449b947061)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:38:58.849056 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae5b910-c465-4366-9b00-18e6a8770136 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:38:58.947731 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-aae5b910-c465-4366-9b00-18e6a8770136 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b811eaeb5434bed909564449b947061', 'request_id': 'req-aae5b910-c465-4366-9b00-18e6a8770136'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:38:59.044745 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-aae5b910-c465-4366-9b00-18e6a8770136 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b811eaeb5434bed909564449b947061', 'request_id': 'req-aae5b910-c465-4366-9b00-18e6a8770136'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:38:59.051527 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-aae5b910-c465-4366-9b00-18e6a8770136 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b811eaeb5434bed909564449b947061', 'request_id': 'req-aae5b910-c465-4366-9b00-18e6a8770136'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:38:59.062270 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-aae5b910-c465-4366-9b00-18e6a8770136 admin admin] The token cache is being invalidate because project 0b811eaeb5434bed909564449b947061 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81731) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:38:59.062562 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-aae5b910-c465-4366-9b00-18e6a8770136 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:38:59.063532 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-aae5b910-c465-4366-9b00-18e6a8770136 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:38:59.065100 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1279/2532] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 24 17:38:58 2022] DELETE /identity/v3/projects/0b811eaeb5434bed909564449b947061 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:38:59.069221 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c93377-59ce-400d-a400-b4b710f36125 service neutron] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:38:59.079480 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c93377-59ce-400d-a400-b4b710f36125 service neutron] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:38:59.135844 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06dee43-cf4a-4162-9fd2-990dd54f755d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:38:59.136473 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06dee43-cf4a-4162-9fd2-990dd54f755d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:38:59.136751 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06dee43-cf4a-4162-9fd2-990dd54f755d admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:38:59.137047 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06dee43-cf4a-4162-9fd2-990dd54f755d admin admin] PATH_INFO: `/v3/users/b9c6cfc4f7d14323a6890c73b30ef608` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:38:59.138162 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06dee43-cf4a-4162-9fd2-990dd54f755d admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9c6cfc4f7d14323a6890c73b30ef608)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:38:59.140143 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06dee43-cf4a-4162-9fd2-990dd54f755d admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:38:59.378213 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-a06dee43-cf4a-4162-9fd2-990dd54f755d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9c6cfc4f7d14323a6890c73b30ef608'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:38:59.450449 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-a06dee43-cf4a-4162-9fd2-990dd54f755d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9c6cfc4f7d14323a6890c73b30ef608'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:38:59.451093 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-a06dee43-cf4a-4162-9fd2-990dd54f755d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9c6cfc4f7d14323a6890c73b30ef608'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:38:59.458508 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-a06dee43-cf4a-4162-9fd2-990dd54f755d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9c6cfc4f7d14323a6890c73b30ef608'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:38:59.459171 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-a06dee43-cf4a-4162-9fd2-990dd54f755d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9c6cfc4f7d14323a6890c73b30ef608'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:38:59.479166 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1254/2533] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 24 17:38:59 2022] DELETE /identity/v3/users/b9c6cfc4f7d14323a6890c73b30ef608 => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:38:59.965921 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae5b910-c465-4366-9b00-18e6a8770136 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:38:59.977411 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-aae5b910-c465-4366-9b00-18e6a8770136 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:38:59.985113 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae5b910-c465-4366-9b00-18e6a8770136 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:00.054970 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232ac963-12ae-4756-828c-1358356b4be2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:00.055968 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232ac963-12ae-4756-828c-1358356b4be2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:00.055968 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232ac963-12ae-4756-828c-1358356b4be2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:00.056396 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232ac963-12ae-4756-828c-1358356b4be2 admin admin] PATH_INFO: `/v3/projects/61b1d4d09e7e47deb59f9802ff305971` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:00.058339 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232ac963-12ae-4756-828c-1358356b4be2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=61b1d4d09e7e47deb59f9802ff305971)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:00.062507 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232ac963-12ae-4756-828c-1358356b4be2 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:00.288670 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-232ac963-12ae-4756-828c-1358356b4be2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '61b1d4d09e7e47deb59f9802ff305971', 'request_id': 'req-232ac963-12ae-4756-828c-1358356b4be2'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:00.387729 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-232ac963-12ae-4756-828c-1358356b4be2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '61b1d4d09e7e47deb59f9802ff305971', 'request_id': 'req-232ac963-12ae-4756-828c-1358356b4be2'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:00.395035 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-232ac963-12ae-4756-828c-1358356b4be2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '61b1d4d09e7e47deb59f9802ff305971', 'request_id': 'req-232ac963-12ae-4756-828c-1358356b4be2'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:00.403732 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:39:00.410316 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-232ac963-12ae-4756-828c-1358356b4be2 admin admin] The token cache is being invalidate because project 61b1d4d09e7e47deb59f9802ff305971 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81731) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:39:00.410615 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-232ac963-12ae-4756-828c-1358356b4be2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:00.411435 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-232ac963-12ae-4756-828c-1358356b4be2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:00.412957 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1280/2534] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 24 17:38:59 2022] DELETE /identity/v3/projects/61b1d4d09e7e47deb59f9802ff305971 => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:39:00.419055 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84039b3-9a27-4163-bbf4-1eeae3d7be27 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:00.419708 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84039b3-9a27-4163-bbf4-1eeae3d7be27 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:00.419708 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84039b3-9a27-4163-bbf4-1eeae3d7be27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:00.439960 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-c84039b3-9a27-4163-bbf4-1eeae3d7be27 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:39:00.441221 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-c84039b3-9a27-4163-bbf4-1eeae3d7be27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:39:00.471077 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:39:00.503792 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1255/2535] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:39:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:39:00.511904 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020b400f-fac7-4307-86dd-1908e6f987ba None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:00.512187 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020b400f-fac7-4307-86dd-1908e6f987ba None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:00.512514 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020b400f-fac7-4307-86dd-1908e6f987ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:00.533976 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-020b400f-fac7-4307-86dd-1908e6f987ba None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:39:00.535218 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-020b400f-fac7-4307-86dd-1908e6f987ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:39:00.549351 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1281/2536] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:39:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:39:00.550328 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84039b3-9a27-4163-bbf4-1eeae3d7be27 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:00.560519 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84039b3-9a27-4163-bbf4-1eeae3d7be27 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:00.565522 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e0b436-3a2d-4b61-945a-3b03f871b76e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:00.566236 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e0b436-3a2d-4b61-945a-3b03f871b76e admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:00.566523 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e0b436-3a2d-4b61-945a-3b03f871b76e admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:00.566799 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e0b436-3a2d-4b61-945a-3b03f871b76e admin admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:00.567487 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e0b436-3a2d-4b61-945a-3b03f871b76e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:39:00.567801 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e0b436-3a2d-4b61-945a-3b03f871b76e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:00.569589 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e0b436-3a2d-4b61-945a-3b03f871b76e admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:00.584264 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1256/2537] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 24 17:39:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:39:00.588535 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020b400f-fac7-4307-86dd-1908e6f987ba None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:00.598773 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020b400f-fac7-4307-86dd-1908e6f987ba None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:00.603995 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5443da-e95a-4424-8fb5-191c23db83c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:00.604690 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5443da-e95a-4424-8fb5-191c23db83c7 admin admin] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:00.605034 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5443da-e95a-4424-8fb5-191c23db83c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:00.605369 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5443da-e95a-4424-8fb5-191c23db83c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:00.606722 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5443da-e95a-4424-8fb5-191c23db83c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:00.608666 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5443da-e95a-4424-8fb5-191c23db83c7 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:00.806164 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1282/2538] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 24 17:39:00 2022] POST /identity/v3/projects => generated 391 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:39:00.811066 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e0b436-3a2d-4b61-945a-3b03f871b76e admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:00.821189 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e0b436-3a2d-4b61-945a-3b03f871b76e admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:00.877059 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abde804-3a89-44b8-8f09-dbb65e17c559 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:00.877668 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abde804-3a89-44b8-8f09-dbb65e17c559 admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:00.877954 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abde804-3a89-44b8-8f09-dbb65e17c559 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:00.878241 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abde804-3a89-44b8-8f09-dbb65e17c559 admin admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:00.878860 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abde804-3a89-44b8-8f09-dbb65e17c559 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:00.880865 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abde804-3a89-44b8-8f09-dbb65e17c559 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:00.999962 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1257/2539] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 24 17:39:00 2022] POST /identity/v3/users => generated 296 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:39:01.003775 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5443da-e95a-4424-8fb5-191c23db83c7 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:01.014281 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5443da-e95a-4424-8fb5-191c23db83c7 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:01.020189 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ad2799-d8fb-4945-ae6f-b9319ca00416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:01.020826 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ad2799-d8fb-4945-ae6f-b9319ca00416 admin admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:01.021110 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ad2799-d8fb-4945-ae6f-b9319ca00416 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:01.021387 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ad2799-d8fb-4945-ae6f-b9319ca00416 admin admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:01.022072 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ad2799-d8fb-4945-ae6f-b9319ca00416 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:01.023720 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ad2799-d8fb-4945-ae6f-b9319ca00416 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:01.035090 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1283/2540] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:39:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:39:01.044297 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abde804-3a89-44b8-8f09-dbb65e17c559 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:01.051313 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abde804-3a89-44b8-8f09-dbb65e17c559 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:01.056274 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea5222c-a47c-446a-b4be-591ca562be83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:01.056829 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea5222c-a47c-446a-b4be-591ca562be83 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:01.057109 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea5222c-a47c-446a-b4be-591ca562be83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:01.057380 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea5222c-a47c-446a-b4be-591ca562be83 admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:01.058056 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea5222c-a47c-446a-b4be-591ca562be83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:01.061229 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea5222c-a47c-446a-b4be-591ca562be83 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:01.075323 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1258/2541] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:39:01 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:39:01.080239 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ad2799-d8fb-4945-ae6f-b9319ca00416 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:01.090867 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ad2799-d8fb-4945-ae6f-b9319ca00416 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:01.096129 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725630a5-3707-4c3c-886f-17956c5e895c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:01.096787 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725630a5-3707-4c3c-886f-17956c5e895c admin admin] REQUEST_METHOD: `PUT` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:01.097085 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725630a5-3707-4c3c-886f-17956c5e895c admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:01.097354 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725630a5-3707-4c3c-886f-17956c5e895c admin admin] PATH_INFO: `/v3/projects/d9e1cd9278fe46a9a2de1e898c4c58a3/users/ec0fb299d5b342e985c42d30d9755baa/roles/60dc7e16aee746d5a3e8aa27d7d8525c` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:01.112895 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725630a5-3707-4c3c-886f-17956c5e895c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9e1cd9278fe46a9a2de1e898c4c58a3, user_id=ec0fb299d5b342e985c42d30d9755baa, role_id=60dc7e16aee746d5a3e8aa27d7d8525c)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:01.115583 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725630a5-3707-4c3c-886f-17956c5e895c admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:01.199118 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1284/2542] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 24 17:39:01 2022] PUT /identity/v3/projects/d9e1cd9278fe46a9a2de1e898c4c58a3/users/ec0fb299d5b342e985c42d30d9755baa/roles/60dc7e16aee746d5a3e8aa27d7d8525c => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:39:01.210012 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6daa04-d9c4-402f-8a0b-ba20d145309e None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:01.210291 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6daa04-d9c4-402f-8a0b-ba20d145309e None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:01.210562 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6daa04-d9c4-402f-8a0b-ba20d145309e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:01.239630 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-4d6daa04-d9c4-402f-8a0b-ba20d145309e None None] MFA Rules not processed for user `ec0fb299d5b342e985c42d30d9755baa`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:39:01.240870 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-4d6daa04-d9c4-402f-8a0b-ba20d145309e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:39:01.306027 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1259/2543] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:39:01 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:39:01.311373 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725630a5-3707-4c3c-886f-17956c5e895c admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:01.381189 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725630a5-3707-4c3c-886f-17956c5e895c admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:01.385975 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fa143c-610b-474c-9fcc-2ccbbf019bef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:01.386582 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fa143c-610b-474c-9fcc-2ccbbf019bef admin admin] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:01.387468 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fa143c-610b-474c-9fcc-2ccbbf019bef admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:01.387746 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fa143c-610b-474c-9fcc-2ccbbf019bef admin admin] PATH_INFO: `/v3/projects` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:01.388369 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fa143c-610b-474c-9fcc-2ccbbf019bef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:01.390274 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fa143c-610b-474c-9fcc-2ccbbf019bef admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:01.566330 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1285/2544] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 24 17:39:01 2022] POST /identity/v3/projects => generated 391 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:39:01.570490 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6daa04-d9c4-402f-8a0b-ba20d145309e None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:01.581413 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6daa04-d9c4-402f-8a0b-ba20d145309e None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:01.608094 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:39:01.641156 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0bc770-a60e-40a9-b9ea-fe3d29e2f68d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:01.641776 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0bc770-a60e-40a9-b9ea-fe3d29e2f68d admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:01.642060 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0bc770-a60e-40a9-b9ea-fe3d29e2f68d admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:01.642351 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0bc770-a60e-40a9-b9ea-fe3d29e2f68d admin admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:01.643012 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0bc770-a60e-40a9-b9ea-fe3d29e2f68d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:01.644940 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0bc770-a60e-40a9-b9ea-fe3d29e2f68d admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:01.782255 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1260/2545] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 24 17:39:01 2022] POST /identity/v3/users => generated 302 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:39:01.801656 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fa143c-610b-474c-9fcc-2ccbbf019bef admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:01.812655 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fa143c-610b-474c-9fcc-2ccbbf019bef admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:01.817316 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc7c77a-9dd2-4367-8393-932a1ab20251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:01.817863 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc7c77a-9dd2-4367-8393-932a1ab20251 admin admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:01.818141 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc7c77a-9dd2-4367-8393-932a1ab20251 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:01.818416 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc7c77a-9dd2-4367-8393-932a1ab20251 admin admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:01.819110 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc7c77a-9dd2-4367-8393-932a1ab20251 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:01.820765 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc7c77a-9dd2-4367-8393-932a1ab20251 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:01.832219 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1286/2546] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:39:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:39:01.842110 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0bc770-a60e-40a9-b9ea-fe3d29e2f68d admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:01.852348 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0bc770-a60e-40a9-b9ea-fe3d29e2f68d admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:01.857571 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35eb8c92-4dec-4326-95e7-bbb224d46792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:01.858256 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35eb8c92-4dec-4326-95e7-bbb224d46792 admin admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:01.858536 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35eb8c92-4dec-4326-95e7-bbb224d46792 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:01.858810 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35eb8c92-4dec-4326-95e7-bbb224d46792 admin admin] PATH_INFO: `/v3/projects/47440a44afa9406e8a12fcf589df1503/users/9ab60136b8294a3aa5ce302d53e40427/roles/c5346bc54abf445f9900edf94264c850` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:01.875382 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35eb8c92-4dec-4326-95e7-bbb224d46792 admin admin] RBAC: Authorizing `identity:create_grant(project_id=47440a44afa9406e8a12fcf589df1503, user_id=9ab60136b8294a3aa5ce302d53e40427, role_id=c5346bc54abf445f9900edf94264c850)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:01.878096 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35eb8c92-4dec-4326-95e7-bbb224d46792 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:01.970917 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1261/2547] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 24 17:39:01 2022] PUT /identity/v3/projects/47440a44afa9406e8a12fcf589df1503/users/9ab60136b8294a3aa5ce302d53e40427/roles/c5346bc54abf445f9900edf94264c850 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:39:01.975072 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc7c77a-9dd2-4367-8393-932a1ab20251 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:01.985714 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc7c77a-9dd2-4367-8393-932a1ab20251 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:02.042987 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d648ebd6-ae45-473f-93e1-2bf964fb60f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:02.044160 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d648ebd6-ae45-473f-93e1-2bf964fb60f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:02.044443 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d648ebd6-ae45-473f-93e1-2bf964fb60f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:02.044715 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d648ebd6-ae45-473f-93e1-2bf964fb60f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:02.045351 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d648ebd6-ae45-473f-93e1-2bf964fb60f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:02.046964 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d648ebd6-ae45-473f-93e1-2bf964fb60f4 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:02.056564 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1287/2548] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:39:01 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:39:02.060844 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35eb8c92-4dec-4326-95e7-bbb224d46792 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:02.071479 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35eb8c92-4dec-4326-95e7-bbb224d46792 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:02.076198 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be890bb-b6e0-49c8-8a1c-f44edf4899d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:02.076860 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be890bb-b6e0-49c8-8a1c-f44edf4899d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:02.077141 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be890bb-b6e0-49c8-8a1c-f44edf4899d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:02.077413 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be890bb-b6e0-49c8-8a1c-f44edf4899d3 admin admin] PATH_INFO: `/v3/projects/47440a44afa9406e8a12fcf589df1503/users/9ab60136b8294a3aa5ce302d53e40427/roles/c5346bc54abf445f9900edf94264c850` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:02.079356 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be890bb-b6e0-49c8-8a1c-f44edf4899d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=47440a44afa9406e8a12fcf589df1503, user_id=9ab60136b8294a3aa5ce302d53e40427, role_id=c5346bc54abf445f9900edf94264c850)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:02.082075 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be890bb-b6e0-49c8-8a1c-f44edf4899d3 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:02.093423 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1262/2549] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 24 17:39:02 2022] PUT /identity/v3/projects/47440a44afa9406e8a12fcf589df1503/users/9ab60136b8294a3aa5ce302d53e40427/roles/c5346bc54abf445f9900edf94264c850 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:39:02.104697 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd48b6ec-59cc-4c40-8fb8-71fb7b348435 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:02.105366 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd48b6ec-59cc-4c40-8fb8-71fb7b348435 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:02.105366 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd48b6ec-59cc-4c40-8fb8-71fb7b348435 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:02.113997 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:39:02.135600 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-cd48b6ec-59cc-4c40-8fb8-71fb7b348435 None None] MFA Rules not processed for user `9ab60136b8294a3aa5ce302d53e40427`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:39:02.137253 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-cd48b6ec-59cc-4c40-8fb8-71fb7b348435 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:39:02.207209 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1288/2550] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:39:02 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:39:02.218611 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be890bb-b6e0-49c8-8a1c-f44edf4899d3 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:02.220521 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-2be890bb-b6e0-49c8-8a1c-f44edf4899d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:39:02.280807 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be890bb-b6e0-49c8-8a1c-f44edf4899d3 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:02.284566 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06904d9-43c2-4650-91a3-6522edd15a41 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be06246d47584b529150ceb98a5cfd95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:02.285180 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06904d9-43c2-4650-91a3-6522edd15a41 service cinder] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:02.285465 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06904d9-43c2-4650-91a3-6522edd15a41 service cinder] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:02.285741 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06904d9-43c2-4650-91a3-6522edd15a41 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:02.295194 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06904d9-43c2-4650-91a3-6522edd15a41 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:02.296629 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06904d9-43c2-4650-91a3-6522edd15a41 service cinder] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:02.306153 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1263/2551] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:39:02 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:39:02.352686 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd48b6ec-59cc-4c40-8fb8-71fb7b348435 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:02.354158 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-cd48b6ec-59cc-4c40-8fb8-71fb7b348435 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:39:02.366315 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd48b6ec-59cc-4c40-8fb8-71fb7b348435 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:02.370021 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d499480a-a834-4a02-a1e3-d542fff4c12a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be06246d47584b529150ceb98a5cfd95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:02.370622 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d499480a-a834-4a02-a1e3-d542fff4c12a service cinder] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:02.370928 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d499480a-a834-4a02-a1e3-d542fff4c12a service cinder] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:02.371253 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d499480a-a834-4a02-a1e3-d542fff4c12a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:02.432392 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d499480a-a834-4a02-a1e3-d542fff4c12a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:02.433845 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d499480a-a834-4a02-a1e3-d542fff4c12a service cinder] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:02.443170 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1289/2552] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:39:02 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:39:06.827356 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06904d9-43c2-4650-91a3-6522edd15a41 service cinder] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:06.828864 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-a06904d9-43c2-4650-91a3-6522edd15a41 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:39:06.902240 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06904d9-43c2-4650-91a3-6522edd15a41 service cinder] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:06.906826 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e0e690-2a11-4611-a2ef-f1b314b1c861 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e9ed4d47aef4140bfb2d3bb798cc559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:06.907505 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e0e690-2a11-4611-a2ef-f1b314b1c861 service neutron] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:06.907782 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e0e690-2a11-4611-a2ef-f1b314b1c861 service neutron] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:06.908057 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e0e690-2a11-4611-a2ef-f1b314b1c861 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:06.909806 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-93e0e690-2a11-4611-a2ef-f1b314b1c861 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:39:06.958237 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:39:07.003079 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e0e690-2a11-4611-a2ef-f1b314b1c861 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:07.004463 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e0e690-2a11-4611-a2ef-f1b314b1c861 service neutron] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:07.025994 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1264/2553] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:39:06 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:39:07.991428 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d499480a-a834-4a02-a1e3-d542fff4c12a service cinder] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:07.992443 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-d499480a-a834-4a02-a1e3-d542fff4c12a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:39:08.109669 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d499480a-a834-4a02-a1e3-d542fff4c12a service cinder] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:08.116449 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b1d9da-9f38-4870-9d5c-ccf0fb1154f5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557aca9255cb4bbb8af52e796c5c5177', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:08.117064 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b1d9da-9f38-4870-9d5c-ccf0fb1154f5 service nova] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:08.117367 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b1d9da-9f38-4870-9d5c-ccf0fb1154f5 service nova] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:08.117628 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b1d9da-9f38-4870-9d5c-ccf0fb1154f5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:08.119407 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-c5b1d9da-9f38-4870-9d5c-ccf0fb1154f5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:39:08.133223 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b1d9da-9f38-4870-9d5c-ccf0fb1154f5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:08.134869 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b1d9da-9f38-4870-9d5c-ccf0fb1154f5 service nova] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:08.147936 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1290/2554] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:39:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:39:08.633168 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e0e690-2a11-4611-a2ef-f1b314b1c861 service neutron] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:08.731932 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e0e690-2a11-4611-a2ef-f1b314b1c861 service neutron] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:08.741813 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b7c3ba-f43d-4fc2-ba28-7c160fb253c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:08.743674 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b7c3ba-f43d-4fc2-ba28-7c160fb253c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:08.743674 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b7c3ba-f43d-4fc2-ba28-7c160fb253c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:08.743674 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b7c3ba-f43d-4fc2-ba28-7c160fb253c5 admin admin] PATH_INFO: `/v3/users/ec0fb299d5b342e985c42d30d9755baa` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:08.744455 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b7c3ba-f43d-4fc2-ba28-7c160fb253c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec0fb299d5b342e985c42d30d9755baa)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:08.748146 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b7c3ba-f43d-4fc2-ba28-7c160fb253c5 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:09.033621 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-20b7c3ba-f43d-4fc2-ba28-7c160fb253c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec0fb299d5b342e985c42d30d9755baa'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:09.108128 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-20b7c3ba-f43d-4fc2-ba28-7c160fb253c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec0fb299d5b342e985c42d30d9755baa'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:09.108801 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-20b7c3ba-f43d-4fc2-ba28-7c160fb253c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec0fb299d5b342e985c42d30d9755baa'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:09.119749 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-20b7c3ba-f43d-4fc2-ba28-7c160fb253c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec0fb299d5b342e985c42d30d9755baa'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:09.120923 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-20b7c3ba-f43d-4fc2-ba28-7c160fb253c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec0fb299d5b342e985c42d30d9755baa'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:09.150167 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1265/2555] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 24 17:39:08 2022] DELETE /identity/v3/users/ec0fb299d5b342e985c42d30d9755baa => generated 0 bytes in 518 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:39:09.157662 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d180b31-de2e-4a13-8084-25b59747c45a None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:09.159894 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d180b31-de2e-4a13-8084-25b59747c45a None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:09.159894 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d180b31-de2e-4a13-8084-25b59747c45a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:09.192321 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-5d180b31-de2e-4a13-8084-25b59747c45a None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:39:09.193744 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-5d180b31-de2e-4a13-8084-25b59747c45a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:39:09.206019 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:39:09.276878 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1291/2556] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:39:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:39:09.287954 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b7c3ba-f43d-4fc2-ba28-7c160fb253c5 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:09.290688 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-20b7c3ba-f43d-4fc2-ba28-7c160fb253c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:39:09.327272 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:39:09.361066 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b7c3ba-f43d-4fc2-ba28-7c160fb253c5 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:09.364806 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7431be33-4b92-4084-a251-69386d3c3a1a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e9ed4d47aef4140bfb2d3bb798cc559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:09.365562 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7431be33-4b92-4084-a251-69386d3c3a1a service neutron] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:09.365758 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7431be33-4b92-4084-a251-69386d3c3a1a service neutron] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:09.366029 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7431be33-4b92-4084-a251-69386d3c3a1a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:09.377940 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7431be33-4b92-4084-a251-69386d3c3a1a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:09.379384 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7431be33-4b92-4084-a251-69386d3c3a1a service neutron] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:09.390606 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1266/2557] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:39:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:39:09.829173 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d180b31-de2e-4a13-8084-25b59747c45a None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:09.833507 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-5d180b31-de2e-4a13-8084-25b59747c45a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:39:09.844472 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d180b31-de2e-4a13-8084-25b59747c45a None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:09.873996 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3d110c-1de7-48a8-92a2-4271fe8ace52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:09.873996 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3d110c-1de7-48a8-92a2-4271fe8ace52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:09.873996 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3d110c-1de7-48a8-92a2-4271fe8ace52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:09.873996 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3d110c-1de7-48a8-92a2-4271fe8ace52 admin admin] PATH_INFO: `/v3/projects/d9e1cd9278fe46a9a2de1e898c4c58a3` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:09.875188 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3d110c-1de7-48a8-92a2-4271fe8ace52 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9e1cd9278fe46a9a2de1e898c4c58a3)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:09.877309 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3d110c-1de7-48a8-92a2-4271fe8ace52 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:10.010464 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-7c3d110c-1de7-48a8-92a2-4271fe8ace52 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9e1cd9278fe46a9a2de1e898c4c58a3', 'request_id': 'req-7c3d110c-1de7-48a8-92a2-4271fe8ace52'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:10.107074 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-7c3d110c-1de7-48a8-92a2-4271fe8ace52 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9e1cd9278fe46a9a2de1e898c4c58a3', 'request_id': 'req-7c3d110c-1de7-48a8-92a2-4271fe8ace52'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:10.115663 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-7c3d110c-1de7-48a8-92a2-4271fe8ace52 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9e1cd9278fe46a9a2de1e898c4c58a3', 'request_id': 'req-7c3d110c-1de7-48a8-92a2-4271fe8ace52'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:10.126299 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-7c3d110c-1de7-48a8-92a2-4271fe8ace52 admin admin] The token cache is being invalidate because project d9e1cd9278fe46a9a2de1e898c4c58a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81731) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:39:10.126624 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-7c3d110c-1de7-48a8-92a2-4271fe8ace52 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:10.127316 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-7c3d110c-1de7-48a8-92a2-4271fe8ace52 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:10.128966 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1292/2558] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 24 17:39:09 2022] DELETE /identity/v3/projects/d9e1cd9278fe46a9a2de1e898c4c58a3 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:39:10.133088 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7431be33-4b92-4084-a251-69386d3c3a1a service neutron] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:10.143935 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7431be33-4b92-4084-a251-69386d3c3a1a service neutron] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:10.210300 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf7e08a-7843-4a88-a971-7ce0f7644333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:10.210967 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf7e08a-7843-4a88-a971-7ce0f7644333 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:10.211252 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf7e08a-7843-4a88-a971-7ce0f7644333 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:10.211532 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf7e08a-7843-4a88-a971-7ce0f7644333 admin admin] PATH_INFO: `/v3/users/9ab60136b8294a3aa5ce302d53e40427` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:10.212674 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf7e08a-7843-4a88-a971-7ce0f7644333 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ab60136b8294a3aa5ce302d53e40427)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:10.215072 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf7e08a-7843-4a88-a971-7ce0f7644333 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:10.459421 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-fdf7e08a-7843-4a88-a971-7ce0f7644333 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ab60136b8294a3aa5ce302d53e40427'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:10.541359 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-fdf7e08a-7843-4a88-a971-7ce0f7644333 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ab60136b8294a3aa5ce302d53e40427'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:10.541978 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-fdf7e08a-7843-4a88-a971-7ce0f7644333 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ab60136b8294a3aa5ce302d53e40427'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:10.550483 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-fdf7e08a-7843-4a88-a971-7ce0f7644333 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ab60136b8294a3aa5ce302d53e40427'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:10.551217 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-fdf7e08a-7843-4a88-a971-7ce0f7644333 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ab60136b8294a3aa5ce302d53e40427'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:10.570823 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1267/2559] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 24 17:39:10 2022] DELETE /identity/v3/users/9ab60136b8294a3aa5ce302d53e40427 => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:39:11.070429 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3d110c-1de7-48a8-92a2-4271fe8ace52 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:11.071563 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-7c3d110c-1de7-48a8-92a2-4271fe8ace52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:39:11.086867 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3d110c-1de7-48a8-92a2-4271fe8ace52 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:11.162479 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c97832-28b1-4e64-a803-97a045c8f3c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:11.162479 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c97832-28b1-4e64-a803-97a045c8f3c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:11.162738 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c97832-28b1-4e64-a803-97a045c8f3c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:11.162841 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c97832-28b1-4e64-a803-97a045c8f3c4 admin admin] PATH_INFO: `/v3/projects/47440a44afa9406e8a12fcf589df1503` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:11.164149 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c97832-28b1-4e64-a803-97a045c8f3c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=47440a44afa9406e8a12fcf589df1503)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:11.166493 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c97832-28b1-4e64-a803-97a045c8f3c4 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:11.267634 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:39:11.274469 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-41c97832-28b1-4e64-a803-97a045c8f3c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '47440a44afa9406e8a12fcf589df1503', 'request_id': 'req-41c97832-28b1-4e64-a803-97a045c8f3c4'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:11.411414 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-41c97832-28b1-4e64-a803-97a045c8f3c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '47440a44afa9406e8a12fcf589df1503', 'request_id': 'req-41c97832-28b1-4e64-a803-97a045c8f3c4'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:11.418757 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-41c97832-28b1-4e64-a803-97a045c8f3c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '47440a44afa9406e8a12fcf589df1503', 'request_id': 'req-41c97832-28b1-4e64-a803-97a045c8f3c4'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:11.429848 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-41c97832-28b1-4e64-a803-97a045c8f3c4 admin admin] The token cache is being invalidate because project 47440a44afa9406e8a12fcf589df1503 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81731) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:39:11.430186 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-41c97832-28b1-4e64-a803-97a045c8f3c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:11.430786 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-41c97832-28b1-4e64-a803-97a045c8f3c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:11.432463 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1293/2560] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 24 17:39:11 2022] DELETE /identity/v3/projects/47440a44afa9406e8a12fcf589df1503 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:39:11.437324 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a948f1-f6bf-439f-85db-e16c95b1d37c None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:11.437834 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a948f1-f6bf-439f-85db-e16c95b1d37c None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:11.437942 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a948f1-f6bf-439f-85db-e16c95b1d37c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:11.458792 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-81a948f1-f6bf-439f-85db-e16c95b1d37c None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:39:11.460096 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-81a948f1-f6bf-439f-85db-e16c95b1d37c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:39:11.506304 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:39:11.521675 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1268/2561] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:39:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:39:11.530853 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eef34a-5508-4a33-835e-893d12742c28 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:11.541149 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eef34a-5508-4a33-835e-893d12742c28 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:11.541149 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eef34a-5508-4a33-835e-893d12742c28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:11.551387 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-e9eef34a-5508-4a33-835e-893d12742c28 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:39:11.552619 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-e9eef34a-5508-4a33-835e-893d12742c28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:39:11.565222 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1294/2562] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:39:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:39:11.569814 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a948f1-f6bf-439f-85db-e16c95b1d37c None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:11.581498 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a948f1-f6bf-439f-85db-e16c95b1d37c None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:11.587365 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2204d3a1-9b53-4d18-b018-38225c759acc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:11.588066 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2204d3a1-9b53-4d18-b018-38225c759acc admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:11.588474 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2204d3a1-9b53-4d18-b018-38225c759acc admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:11.588768 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2204d3a1-9b53-4d18-b018-38225c759acc admin admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:11.589426 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2204d3a1-9b53-4d18-b018-38225c759acc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:39:11.589751 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2204d3a1-9b53-4d18-b018-38225c759acc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:11.592104 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2204d3a1-9b53-4d18-b018-38225c759acc admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:11.611322 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1269/2563] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 24 17:39:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:39:11.621568 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9eef34a-5508-4a33-835e-893d12742c28 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:11.632900 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9eef34a-5508-4a33-835e-893d12742c28 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:11.639540 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab65f81a-fbc9-4938-9f30-bb011650ae1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:11.640437 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab65f81a-fbc9-4938-9f30-bb011650ae1b admin admin] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:11.640795 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab65f81a-fbc9-4938-9f30-bb011650ae1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:11.641126 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab65f81a-fbc9-4938-9f30-bb011650ae1b admin admin] PATH_INFO: `/v3/projects` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:11.641897 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab65f81a-fbc9-4938-9f30-bb011650ae1b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:11.644707 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab65f81a-fbc9-4938-9f30-bb011650ae1b admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:11.748821 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1295/2564] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 24 17:39:11 2022] POST /identity/v3/projects => generated 401 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:39:11.753402 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2204d3a1-9b53-4d18-b018-38225c759acc admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:11.763997 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2204d3a1-9b53-4d18-b018-38225c759acc admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:11.836231 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9824d52d-2e46-41b0-b169-e847526665dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:11.836872 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9824d52d-2e46-41b0-b169-e847526665dd admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:11.837412 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9824d52d-2e46-41b0-b169-e847526665dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:11.837910 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9824d52d-2e46-41b0-b169-e847526665dd admin admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:11.839191 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9824d52d-2e46-41b0-b169-e847526665dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:11.841647 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9824d52d-2e46-41b0-b169-e847526665dd admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:11.947834 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1270/2565] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 24 17:39:11 2022] POST /identity/v3/users => generated 301 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:39:11.951933 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab65f81a-fbc9-4938-9f30-bb011650ae1b admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:11.963456 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab65f81a-fbc9-4938-9f30-bb011650ae1b admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:11.970303 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457e5b59-ce49-4e96-828a-a9d365c9071b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:11.970951 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457e5b59-ce49-4e96-828a-a9d365c9071b admin admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:11.979747 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457e5b59-ce49-4e96-828a-a9d365c9071b admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:11.979747 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457e5b59-ce49-4e96-828a-a9d365c9071b admin admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:11.979747 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457e5b59-ce49-4e96-828a-a9d365c9071b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:11.979747 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457e5b59-ce49-4e96-828a-a9d365c9071b admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:11.986252 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1296/2566] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:39:11 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:39:11.990361 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9824d52d-2e46-41b0-b169-e847526665dd admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:12.001436 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9824d52d-2e46-41b0-b169-e847526665dd admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:12.006933 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d18bce4-b690-49b6-893c-79bb12f28491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:12.007574 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d18bce4-b690-49b6-893c-79bb12f28491 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:12.007885 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d18bce4-b690-49b6-893c-79bb12f28491 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:12.008160 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d18bce4-b690-49b6-893c-79bb12f28491 admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:12.008817 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d18bce4-b690-49b6-893c-79bb12f28491 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:12.010457 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d18bce4-b690-49b6-893c-79bb12f28491 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:12.024734 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1271/2567] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:39:11 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:39:12.029572 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457e5b59-ce49-4e96-828a-a9d365c9071b admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:12.040611 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457e5b59-ce49-4e96-828a-a9d365c9071b admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:12.045459 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cdfb3e-da50-49dc-bba5-d84f0c947425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:12.046133 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cdfb3e-da50-49dc-bba5-d84f0c947425 admin admin] REQUEST_METHOD: `PUT` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:12.046418 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cdfb3e-da50-49dc-bba5-d84f0c947425 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:12.046688 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cdfb3e-da50-49dc-bba5-d84f0c947425 admin admin] PATH_INFO: `/v3/projects/5db9c04728b74780bab0f73d0bebb43e/users/596606c665cd432db2ae7bf9a6cf51c3/roles/60dc7e16aee746d5a3e8aa27d7d8525c` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:12.065862 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cdfb3e-da50-49dc-bba5-d84f0c947425 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5db9c04728b74780bab0f73d0bebb43e, user_id=596606c665cd432db2ae7bf9a6cf51c3, role_id=60dc7e16aee746d5a3e8aa27d7d8525c)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:12.070373 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cdfb3e-da50-49dc-bba5-d84f0c947425 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:12.154146 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1297/2568] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 24 17:39:12 2022] PUT /identity/v3/projects/5db9c04728b74780bab0f73d0bebb43e/users/596606c665cd432db2ae7bf9a6cf51c3/roles/60dc7e16aee746d5a3e8aa27d7d8525c => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:39:12.166125 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108c9024-3751-48c1-8a35-b224ee96a0bd None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:12.166404 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108c9024-3751-48c1-8a35-b224ee96a0bd None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:12.166678 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108c9024-3751-48c1-8a35-b224ee96a0bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:12.195892 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-108c9024-3751-48c1-8a35-b224ee96a0bd None None] MFA Rules not processed for user `596606c665cd432db2ae7bf9a6cf51c3`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:39:12.197183 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-108c9024-3751-48c1-8a35-b224ee96a0bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:39:12.266317 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1272/2569] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:39:12 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:39:12.272018 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cdfb3e-da50-49dc-bba5-d84f0c947425 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:12.350148 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cdfb3e-da50-49dc-bba5-d84f0c947425 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:12.354979 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b4c94e-2c34-40d6-ac14-461c88830429 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:12.361293 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b4c94e-2c34-40d6-ac14-461c88830429 admin admin] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:12.361293 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b4c94e-2c34-40d6-ac14-461c88830429 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:12.361293 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b4c94e-2c34-40d6-ac14-461c88830429 admin admin] PATH_INFO: `/v3/projects` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:12.361293 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b4c94e-2c34-40d6-ac14-461c88830429 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:12.361293 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b4c94e-2c34-40d6-ac14-461c88830429 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:12.468531 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1298/2570] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 24 17:39:12 2022] POST /identity/v3/projects => generated 401 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:39:12.472381 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108c9024-3751-48c1-8a35-b224ee96a0bd None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:12.483183 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108c9024-3751-48c1-8a35-b224ee96a0bd None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:12.547482 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:39:12.569097 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7fd0c0-b872-4113-b140-0c70729063d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:12.574239 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7fd0c0-b872-4113-b140-0c70729063d0 admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:12.574239 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7fd0c0-b872-4113-b140-0c70729063d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:12.574239 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7fd0c0-b872-4113-b140-0c70729063d0 admin admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:12.574239 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7fd0c0-b872-4113-b140-0c70729063d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:12.574239 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7fd0c0-b872-4113-b140-0c70729063d0 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:12.673789 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1273/2571] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 24 17:39:12 2022] POST /identity/v3/users => generated 307 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:39:12.679630 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b4c94e-2c34-40d6-ac14-461c88830429 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:12.695615 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b4c94e-2c34-40d6-ac14-461c88830429 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:12.703918 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a411b0-8c43-4e08-b4a4-0708bd2f0137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:12.704984 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a411b0-8c43-4e08-b4a4-0708bd2f0137 admin admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:12.705481 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a411b0-8c43-4e08-b4a4-0708bd2f0137 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:12.705950 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a411b0-8c43-4e08-b4a4-0708bd2f0137 admin admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:12.706746 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a411b0-8c43-4e08-b4a4-0708bd2f0137 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:12.710490 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a411b0-8c43-4e08-b4a4-0708bd2f0137 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:12.726285 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1299/2572] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:39:12 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:39:12.732337 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7fd0c0-b872-4113-b140-0c70729063d0 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:12.742435 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7fd0c0-b872-4113-b140-0c70729063d0 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:12.748345 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b7e2b4-98d8-49b7-a82e-095191cbbfb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:12.749075 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b7e2b4-98d8-49b7-a82e-095191cbbfb2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:12.749397 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b7e2b4-98d8-49b7-a82e-095191cbbfb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:12.749725 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b7e2b4-98d8-49b7-a82e-095191cbbfb2 admin admin] PATH_INFO: `/v3/projects/8249533529a546628870f4fdad68433e/users/1006fcb736bd46b48401b57aea0c1990/roles/c5346bc54abf445f9900edf94264c850` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:12.773139 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b7e2b4-98d8-49b7-a82e-095191cbbfb2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8249533529a546628870f4fdad68433e, user_id=1006fcb736bd46b48401b57aea0c1990, role_id=c5346bc54abf445f9900edf94264c850)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:12.776103 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b7e2b4-98d8-49b7-a82e-095191cbbfb2 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:12.892625 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1274/2573] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 24 17:39:12 2022] PUT /identity/v3/projects/8249533529a546628870f4fdad68433e/users/1006fcb736bd46b48401b57aea0c1990/roles/c5346bc54abf445f9900edf94264c850 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:39:12.896900 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a411b0-8c43-4e08-b4a4-0708bd2f0137 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:12.908510 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a411b0-8c43-4e08-b4a4-0708bd2f0137 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:12.943045 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:39:12.971831 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc92dc70-29dc-4a32-ab57-43c0cdc801ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:12.973297 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc92dc70-29dc-4a32-ab57-43c0cdc801ff admin admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:12.973671 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc92dc70-29dc-4a32-ab57-43c0cdc801ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:12.974072 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc92dc70-29dc-4a32-ab57-43c0cdc801ff admin admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:12.975545 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc92dc70-29dc-4a32-ab57-43c0cdc801ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:12.977374 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc92dc70-29dc-4a32-ab57-43c0cdc801ff admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:12.989345 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1300/2574] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:39:12 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:39:12.993495 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b7e2b4-98d8-49b7-a82e-095191cbbfb2 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:13.005520 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b7e2b4-98d8-49b7-a82e-095191cbbfb2 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:13.010468 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360aa4cb-cff0-4d27-80b9-e4707828e348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:13.011170 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360aa4cb-cff0-4d27-80b9-e4707828e348 admin admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:13.011451 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360aa4cb-cff0-4d27-80b9-e4707828e348 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:13.011814 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360aa4cb-cff0-4d27-80b9-e4707828e348 admin admin] PATH_INFO: `/v3/projects/8249533529a546628870f4fdad68433e/users/1006fcb736bd46b48401b57aea0c1990/roles/c5346bc54abf445f9900edf94264c850` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:13.013755 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360aa4cb-cff0-4d27-80b9-e4707828e348 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8249533529a546628870f4fdad68433e, user_id=1006fcb736bd46b48401b57aea0c1990, role_id=c5346bc54abf445f9900edf94264c850)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:13.016498 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360aa4cb-cff0-4d27-80b9-e4707828e348 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:13.031359 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1275/2575] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 24 17:39:12 2022] PUT /identity/v3/projects/8249533529a546628870f4fdad68433e/users/1006fcb736bd46b48401b57aea0c1990/roles/c5346bc54abf445f9900edf94264c850 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:39:13.041906 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2a5c61-fc0f-45b0-8bf1-9ea2c0fb4580 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:13.042306 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2a5c61-fc0f-45b0-8bf1-9ea2c0fb4580 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:13.042460 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2a5c61-fc0f-45b0-8bf1-9ea2c0fb4580 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:13.081851 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-3c2a5c61-fc0f-45b0-8bf1-9ea2c0fb4580 None None] MFA Rules not processed for user `1006fcb736bd46b48401b57aea0c1990`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:39:13.084081 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-3c2a5c61-fc0f-45b0-8bf1-9ea2c0fb4580 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:39:13.167917 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1301/2576] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:39:13 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:39:13.181366 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360aa4cb-cff0-4d27-80b9-e4707828e348 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:13.184378 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-360aa4cb-cff0-4d27-80b9-e4707828e348 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:39:13.255541 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360aa4cb-cff0-4d27-80b9-e4707828e348 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:13.259401 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398970d9-03e9-450c-8d78-28e8637f6550 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be06246d47584b529150ceb98a5cfd95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:13.260132 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398970d9-03e9-450c-8d78-28e8637f6550 service cinder] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:13.260415 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398970d9-03e9-450c-8d78-28e8637f6550 service cinder] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:13.260684 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398970d9-03e9-450c-8d78-28e8637f6550 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:13.270563 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398970d9-03e9-450c-8d78-28e8637f6550 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:13.272108 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398970d9-03e9-450c-8d78-28e8637f6550 service cinder] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:13.283216 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1276/2577] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:39:13 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:39:13.471241 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2a5c61-fc0f-45b0-8bf1-9ea2c0fb4580 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:13.481506 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2a5c61-fc0f-45b0-8bf1-9ea2c0fb4580 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:13.487209 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bdba8d-66d1-4fdc-a833-9980d6cb269e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be06246d47584b529150ceb98a5cfd95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:13.487864 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bdba8d-66d1-4fdc-a833-9980d6cb269e service cinder] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:13.488143 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bdba8d-66d1-4fdc-a833-9980d6cb269e service cinder] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:13.488447 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bdba8d-66d1-4fdc-a833-9980d6cb269e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:13.553128 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bdba8d-66d1-4fdc-a833-9980d6cb269e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:13.554648 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bdba8d-66d1-4fdc-a833-9980d6cb269e service cinder] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:13.565470 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1302/2578] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:39:13 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:39:19.914551 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398970d9-03e9-450c-8d78-28e8637f6550 service cinder] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:19.985026 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398970d9-03e9-450c-8d78-28e8637f6550 service cinder] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:19.990079 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989b602e-5efd-448f-81e9-26ed71fb8a9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:19.990846 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989b602e-5efd-448f-81e9-26ed71fb8a9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:19.991981 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989b602e-5efd-448f-81e9-26ed71fb8a9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:19.991981 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989b602e-5efd-448f-81e9-26ed71fb8a9e admin admin] PATH_INFO: `/v3/users/596606c665cd432db2ae7bf9a6cf51c3` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:19.992737 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989b602e-5efd-448f-81e9-26ed71fb8a9e admin admin] RBAC: Authorizing `identity:delete_user(user_id=596606c665cd432db2ae7bf9a6cf51c3)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:19.994666 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989b602e-5efd-448f-81e9-26ed71fb8a9e admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:20.207477 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=89,keystone:INSERT=5,keystone:DELETE=6 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:39:20.213065 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-989b602e-5efd-448f-81e9-26ed71fb8a9e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '596606c665cd432db2ae7bf9a6cf51c3'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:20.312773 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-989b602e-5efd-448f-81e9-26ed71fb8a9e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '596606c665cd432db2ae7bf9a6cf51c3'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:20.313183 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-989b602e-5efd-448f-81e9-26ed71fb8a9e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '596606c665cd432db2ae7bf9a6cf51c3'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:20.321942 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-989b602e-5efd-448f-81e9-26ed71fb8a9e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '596606c665cd432db2ae7bf9a6cf51c3'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:20.322753 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-989b602e-5efd-448f-81e9-26ed71fb8a9e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '596606c665cd432db2ae7bf9a6cf51c3'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:20.340701 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1277/2579] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 24 17:39:19 2022] DELETE /identity/v3/users/596606c665cd432db2ae7bf9a6cf51c3 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:39:20.345051 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd45008-1d99-4e5b-956f-8d7f239af475 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:20.345551 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd45008-1d99-4e5b-956f-8d7f239af475 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:20.345668 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd45008-1d99-4e5b-956f-8d7f239af475 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:20.368814 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-fdd45008-1d99-4e5b-956f-8d7f239af475 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:39:20.370058 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-fdd45008-1d99-4e5b-956f-8d7f239af475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:39:20.431920 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:39:20.444719 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1303/2580] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:39:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:39:20.455301 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989b602e-5efd-448f-81e9-26ed71fb8a9e admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:20.456833 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-989b602e-5efd-448f-81e9-26ed71fb8a9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:39:20.528627 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989b602e-5efd-448f-81e9-26ed71fb8a9e admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:20.532910 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de879f1-419d-405c-b717-cb0970493993 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e9ed4d47aef4140bfb2d3bb798cc559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:20.533598 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de879f1-419d-405c-b717-cb0970493993 service neutron] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:20.533842 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de879f1-419d-405c-b717-cb0970493993 service neutron] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:20.534112 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de879f1-419d-405c-b717-cb0970493993 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:20.546926 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de879f1-419d-405c-b717-cb0970493993 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:20.549415 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de879f1-419d-405c-b717-cb0970493993 service neutron] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:20.569879 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1278/2581] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:39:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:39:21.126978 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd45008-1d99-4e5b-956f-8d7f239af475 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:21.128690 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-fdd45008-1d99-4e5b-956f-8d7f239af475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:39:21.149975 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd45008-1d99-4e5b-956f-8d7f239af475 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:21.159530 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f20702-2d53-465e-92f1-a5cdf04135ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:21.160141 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f20702-2d53-465e-92f1-a5cdf04135ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:21.160424 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f20702-2d53-465e-92f1-a5cdf04135ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:21.160699 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f20702-2d53-465e-92f1-a5cdf04135ad admin admin] PATH_INFO: `/v3/projects/5db9c04728b74780bab0f73d0bebb43e` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:21.161910 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f20702-2d53-465e-92f1-a5cdf04135ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=5db9c04728b74780bab0f73d0bebb43e)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:21.163875 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f20702-2d53-465e-92f1-a5cdf04135ad admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:21.292827 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-f9f20702-2d53-465e-92f1-a5cdf04135ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5db9c04728b74780bab0f73d0bebb43e', 'request_id': 'req-f9f20702-2d53-465e-92f1-a5cdf04135ad'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:21.352879 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-f9f20702-2d53-465e-92f1-a5cdf04135ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5db9c04728b74780bab0f73d0bebb43e', 'request_id': 'req-f9f20702-2d53-465e-92f1-a5cdf04135ad'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:21.361081 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-f9f20702-2d53-465e-92f1-a5cdf04135ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5db9c04728b74780bab0f73d0bebb43e', 'request_id': 'req-f9f20702-2d53-465e-92f1-a5cdf04135ad'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:21.372735 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-f9f20702-2d53-465e-92f1-a5cdf04135ad admin admin] The token cache is being invalidate because project 5db9c04728b74780bab0f73d0bebb43e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81731) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:39:21.372944 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-f9f20702-2d53-465e-92f1-a5cdf04135ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:21.374042 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-f9f20702-2d53-465e-92f1-a5cdf04135ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:21.375299 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1304/2582] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 24 17:39:21 2022] DELETE /identity/v3/projects/5db9c04728b74780bab0f73d0bebb43e => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:39:21.380011 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de879f1-419d-405c-b717-cb0970493993 service neutron] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:21.391106 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de879f1-419d-405c-b717-cb0970493993 service neutron] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:21.462474 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301e7ecf-b67e-40d9-b5bd-06e1df9d8a75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:21.463075 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301e7ecf-b67e-40d9-b5bd-06e1df9d8a75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:21.463354 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301e7ecf-b67e-40d9-b5bd-06e1df9d8a75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:21.463639 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301e7ecf-b67e-40d9-b5bd-06e1df9d8a75 admin admin] PATH_INFO: `/v3/users/1006fcb736bd46b48401b57aea0c1990` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:21.464811 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301e7ecf-b67e-40d9-b5bd-06e1df9d8a75 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1006fcb736bd46b48401b57aea0c1990)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:21.466756 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301e7ecf-b67e-40d9-b5bd-06e1df9d8a75 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:21.653226 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-301e7ecf-b67e-40d9-b5bd-06e1df9d8a75 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1006fcb736bd46b48401b57aea0c1990'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:21.736100 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-301e7ecf-b67e-40d9-b5bd-06e1df9d8a75 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1006fcb736bd46b48401b57aea0c1990'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:21.736758 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-301e7ecf-b67e-40d9-b5bd-06e1df9d8a75 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1006fcb736bd46b48401b57aea0c1990'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:21.744521 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-301e7ecf-b67e-40d9-b5bd-06e1df9d8a75 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1006fcb736bd46b48401b57aea0c1990'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:21.745167 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-301e7ecf-b67e-40d9-b5bd-06e1df9d8a75 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1006fcb736bd46b48401b57aea0c1990'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:21.747235 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:39:21.763768 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1279/2583] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 24 17:39:21 2022] DELETE /identity/v3/users/1006fcb736bd46b48401b57aea0c1990 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:39:22.084919 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f20702-2d53-465e-92f1-a5cdf04135ad admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:22.086364 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-f9f20702-2d53-465e-92f1-a5cdf04135ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:39:22.147202 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f20702-2d53-465e-92f1-a5cdf04135ad admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:22.196535 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e75d3a0-751a-4497-b999-64e4217c8abc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e9ed4d47aef4140bfb2d3bb798cc559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:22.197149 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e75d3a0-751a-4497-b999-64e4217c8abc service neutron] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:22.197426 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e75d3a0-751a-4497-b999-64e4217c8abc service neutron] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:22.197699 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e75d3a0-751a-4497-b999-64e4217c8abc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:22.199461 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-7e75d3a0-751a-4497-b999-64e4217c8abc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:39:22.206049 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301e7ecf-b67e-40d9-b5bd-06e1df9d8a75 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:22.208389 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-301e7ecf-b67e-40d9-b5bd-06e1df9d8a75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:39:22.259879 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:39:22.275364 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e75d3a0-751a-4497-b999-64e4217c8abc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:22.277221 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e75d3a0-751a-4497-b999-64e4217c8abc service neutron] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:22.283826 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301e7ecf-b67e-40d9-b5bd-06e1df9d8a75 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:22.287667 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1305/2584] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:39:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:39:22.288532 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc4498a-f69d-4191-92a1-01ec2bb2ecea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:22.289239 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc4498a-f69d-4191-92a1-01ec2bb2ecea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:22.289526 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc4498a-f69d-4191-92a1-01ec2bb2ecea admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:22.289799 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc4498a-f69d-4191-92a1-01ec2bb2ecea admin admin] PATH_INFO: `/v3/projects/8249533529a546628870f4fdad68433e` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:22.290991 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc4498a-f69d-4191-92a1-01ec2bb2ecea admin admin] RBAC: Authorizing `identity:delete_project(project_id=8249533529a546628870f4fdad68433e)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:22.292947 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc4498a-f69d-4191-92a1-01ec2bb2ecea admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:22.398187 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-4fc4498a-f69d-4191-92a1-01ec2bb2ecea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8249533529a546628870f4fdad68433e', 'request_id': 'req-4fc4498a-f69d-4191-92a1-01ec2bb2ecea'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:22.491539 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-4fc4498a-f69d-4191-92a1-01ec2bb2ecea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8249533529a546628870f4fdad68433e', 'request_id': 'req-4fc4498a-f69d-4191-92a1-01ec2bb2ecea'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:22.498597 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-4fc4498a-f69d-4191-92a1-01ec2bb2ecea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8249533529a546628870f4fdad68433e', 'request_id': 'req-4fc4498a-f69d-4191-92a1-01ec2bb2ecea'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:22.511618 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-4fc4498a-f69d-4191-92a1-01ec2bb2ecea admin admin] The token cache is being invalidate because project 8249533529a546628870f4fdad68433e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81732) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:39:22.526792 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-4fc4498a-f69d-4191-92a1-01ec2bb2ecea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:22.526792 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-4fc4498a-f69d-4191-92a1-01ec2bb2ecea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:22.526792 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1280/2585] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 24 17:39:22 2022] DELETE /identity/v3/projects/8249533529a546628870f4fdad68433e => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:39:22.527130 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37180cde-de60-4e92-90b3-6f6bd7d52b39 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:22.527130 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37180cde-de60-4e92-90b3-6f6bd7d52b39 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:22.527130 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37180cde-de60-4e92-90b3-6f6bd7d52b39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:22.551160 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-37180cde-de60-4e92-90b3-6f6bd7d52b39 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:39:22.551942 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-37180cde-de60-4e92-90b3-6f6bd7d52b39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:39:22.631328 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1306/2586] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:39:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:39:22.639431 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549d5faa-7a90-445e-8787-489139937f9b None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:22.639714 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549d5faa-7a90-445e-8787-489139937f9b None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:22.640411 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549d5faa-7a90-445e-8787-489139937f9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:22.663737 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-549d5faa-7a90-445e-8787-489139937f9b None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:39:22.664946 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-549d5faa-7a90-445e-8787-489139937f9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:39:22.673658 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1281/2587] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:39:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:39:22.678425 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37180cde-de60-4e92-90b3-6f6bd7d52b39 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:22.689958 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37180cde-de60-4e92-90b3-6f6bd7d52b39 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:22.695785 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e8ba7d-3250-444d-8842-ccdb3167e50f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:22.696623 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e8ba7d-3250-444d-8842-ccdb3167e50f admin admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:22.697065 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e8ba7d-3250-444d-8842-ccdb3167e50f admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:22.697520 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e8ba7d-3250-444d-8842-ccdb3167e50f admin admin] PATH_INFO: `/v3/domains` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:22.698430 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e8ba7d-3250-444d-8842-ccdb3167e50f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:39:22.699196 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e8ba7d-3250-444d-8842-ccdb3167e50f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:22.701503 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e8ba7d-3250-444d-8842-ccdb3167e50f admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:22.716882 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1307/2588] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 24 17:39:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:39:22.721711 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549d5faa-7a90-445e-8787-489139937f9b None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:22.732841 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549d5faa-7a90-445e-8787-489139937f9b None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:22.738458 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cc8703-4596-4618-842b-01499d4d3102 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:22.739552 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cc8703-4596-4618-842b-01499d4d3102 admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:22.739932 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cc8703-4596-4618-842b-01499d4d3102 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:22.740300 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cc8703-4596-4618-842b-01499d4d3102 admin admin] PATH_INFO: `/v3/projects` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:22.741039 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cc8703-4596-4618-842b-01499d4d3102 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:22.743102 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cc8703-4596-4618-842b-01499d4d3102 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:22.876903 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1282/2589] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 24 17:39:22 2022] POST /identity/v3/projects => generated 377 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:39:22.880842 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e8ba7d-3250-444d-8842-ccdb3167e50f admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:22.891558 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e8ba7d-3250-444d-8842-ccdb3167e50f admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:22.977849 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6818753-7eff-4872-ac0c-0367063783f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:22.978998 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6818753-7eff-4872-ac0c-0367063783f3 admin admin] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:22.979474 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6818753-7eff-4872-ac0c-0367063783f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:22.985097 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6818753-7eff-4872-ac0c-0367063783f3 admin admin] PATH_INFO: `/v3/users` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:22.985097 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6818753-7eff-4872-ac0c-0367063783f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:22.985097 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6818753-7eff-4872-ac0c-0367063783f3 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:23.101970 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1308/2590] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 24 17:39:22 2022] POST /identity/v3/users => generated 289 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:39:23.106244 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cc8703-4596-4618-842b-01499d4d3102 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:23.118381 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cc8703-4596-4618-842b-01499d4d3102 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:23.123983 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be5736e-93e9-4f5f-9dc5-05c22572a97c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:23.124604 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be5736e-93e9-4f5f-9dc5-05c22572a97c admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:23.124884 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be5736e-93e9-4f5f-9dc5-05c22572a97c admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:23.125156 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be5736e-93e9-4f5f-9dc5-05c22572a97c admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:23.125802 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be5736e-93e9-4f5f-9dc5-05c22572a97c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:23.127883 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be5736e-93e9-4f5f-9dc5-05c22572a97c admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:23.138644 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1283/2591] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:39:23 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:39:23.142416 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6818753-7eff-4872-ac0c-0367063783f3 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:23.152837 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6818753-7eff-4872-ac0c-0367063783f3 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:23.160048 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7844828b-15d3-40a9-aa49-fffccc9fb4b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:23.160703 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7844828b-15d3-40a9-aa49-fffccc9fb4b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:23.160994 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7844828b-15d3-40a9-aa49-fffccc9fb4b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:23.161317 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7844828b-15d3-40a9-aa49-fffccc9fb4b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:23.162003 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7844828b-15d3-40a9-aa49-fffccc9fb4b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:23.163641 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7844828b-15d3-40a9-aa49-fffccc9fb4b8 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:23.176346 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1309/2592] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:39:23 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:39:23.180723 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be5736e-93e9-4f5f-9dc5-05c22572a97c admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:23.190659 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be5736e-93e9-4f5f-9dc5-05c22572a97c admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:23.195262 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e6755a-a7de-4bb5-9cac-ebb215c9876e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:23.195910 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e6755a-a7de-4bb5-9cac-ebb215c9876e admin admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:23.196187 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e6755a-a7de-4bb5-9cac-ebb215c9876e admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:23.196456 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e6755a-a7de-4bb5-9cac-ebb215c9876e admin admin] PATH_INFO: `/v3/projects/1a43e26116624ddc98cc48f4e799c9bb/users/3c094b49a6df4b72a9b9321f7fae2031/roles/60dc7e16aee746d5a3e8aa27d7d8525c` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:23.212622 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e6755a-a7de-4bb5-9cac-ebb215c9876e admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a43e26116624ddc98cc48f4e799c9bb, user_id=3c094b49a6df4b72a9b9321f7fae2031, role_id=60dc7e16aee746d5a3e8aa27d7d8525c)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:23.215366 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e6755a-a7de-4bb5-9cac-ebb215c9876e admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:23.318848 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1284/2593] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 24 17:39:23 2022] PUT /identity/v3/projects/1a43e26116624ddc98cc48f4e799c9bb/users/3c094b49a6df4b72a9b9321f7fae2031/roles/60dc7e16aee746d5a3e8aa27d7d8525c => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:39:23.320888 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6e5720-cbf1-491e-a4a8-337e00579323 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:23.321250 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6e5720-cbf1-491e-a4a8-337e00579323 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:23.321583 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6e5720-cbf1-491e-a4a8-337e00579323 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:23.355485 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-6b6e5720-cbf1-491e-a4a8-337e00579323 None None] MFA Rules not processed for user `3c094b49a6df4b72a9b9321f7fae2031`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:39:23.357016 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-6b6e5720-cbf1-491e-a4a8-337e00579323 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:39:23.434826 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:39:23.468849 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1310/2594] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:39:23 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:39:23.473969 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e6755a-a7de-4bb5-9cac-ebb215c9876e admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:23.488626 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=88,keystone:DELETE=9,keystone:INSERT=3 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:39:23.598777 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e6755a-a7de-4bb5-9cac-ebb215c9876e admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:23.598777 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60d38ee-7e62-4c42-ad21-882cd0d3a5dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:23.598777 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60d38ee-7e62-4c42-ad21-882cd0d3a5dd admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:23.598777 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60d38ee-7e62-4c42-ad21-882cd0d3a5dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:23.599187 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60d38ee-7e62-4c42-ad21-882cd0d3a5dd admin admin] PATH_INFO: `/v3/projects` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:23.599187 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60d38ee-7e62-4c42-ad21-882cd0d3a5dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:23.600104 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60d38ee-7e62-4c42-ad21-882cd0d3a5dd admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:23.686628 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1285/2595] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 24 17:39:23 2022] POST /identity/v3/projects => generated 377 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:39:23.692011 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6e5720-cbf1-491e-a4a8-337e00579323 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:23.703320 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6e5720-cbf1-491e-a4a8-337e00579323 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:23.760739 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64ea9c4-c18b-46a3-a4be-71e516f16006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:23.761362 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64ea9c4-c18b-46a3-a4be-71e516f16006 admin admin] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:23.761645 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64ea9c4-c18b-46a3-a4be-71e516f16006 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:23.761923 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64ea9c4-c18b-46a3-a4be-71e516f16006 admin admin] PATH_INFO: `/v3/users` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:23.762688 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64ea9c4-c18b-46a3-a4be-71e516f16006 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:23.765365 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64ea9c4-c18b-46a3-a4be-71e516f16006 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:23.845229 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1311/2596] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 24 17:39:23 2022] POST /identity/v3/users => generated 289 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:39:23.849361 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60d38ee-7e62-4c42-ad21-882cd0d3a5dd admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:23.861048 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60d38ee-7e62-4c42-ad21-882cd0d3a5dd admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:23.867245 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef315258-7de4-4340-919a-adec59077613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:23.867948 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef315258-7de4-4340-919a-adec59077613 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:23.868299 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef315258-7de4-4340-919a-adec59077613 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:23.868648 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef315258-7de4-4340-919a-adec59077613 admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:23.869306 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef315258-7de4-4340-919a-adec59077613 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:23.871001 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef315258-7de4-4340-919a-adec59077613 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:23.883601 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1286/2597] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:39:23 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:39:23.888442 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64ea9c4-c18b-46a3-a4be-71e516f16006 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:23.900835 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64ea9c4-c18b-46a3-a4be-71e516f16006 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:23.906478 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b0860c-e84f-498a-8467-36c267adc6cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:23.907198 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b0860c-e84f-498a-8467-36c267adc6cd admin admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:23.907481 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b0860c-e84f-498a-8467-36c267adc6cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:23.907756 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b0860c-e84f-498a-8467-36c267adc6cd admin admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:23.908468 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b0860c-e84f-498a-8467-36c267adc6cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:23.910121 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b0860c-e84f-498a-8467-36c267adc6cd admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:23.926123 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1312/2598] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:39:23 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:39:23.927348 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef315258-7de4-4340-919a-adec59077613 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:23.938692 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef315258-7de4-4340-919a-adec59077613 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:23.943850 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461fada2-5b8c-493c-a5fb-fae34af5b304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:23.944513 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461fada2-5b8c-493c-a5fb-fae34af5b304 admin admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:23.944790 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461fada2-5b8c-493c-a5fb-fae34af5b304 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:23.945074 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461fada2-5b8c-493c-a5fb-fae34af5b304 admin admin] PATH_INFO: `/v3/projects/aca526fc4af5431581ac05a15d035a9d/users/a2a95d1c2e174d3b8c4f2e18aaf4a380/roles/60dc7e16aee746d5a3e8aa27d7d8525c` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:23.962701 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461fada2-5b8c-493c-a5fb-fae34af5b304 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aca526fc4af5431581ac05a15d035a9d, user_id=a2a95d1c2e174d3b8c4f2e18aaf4a380, role_id=60dc7e16aee746d5a3e8aa27d7d8525c)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:23.967035 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461fada2-5b8c-493c-a5fb-fae34af5b304 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:24.029162 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1287/2599] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 24 17:39:23 2022] PUT /identity/v3/projects/aca526fc4af5431581ac05a15d035a9d/users/a2a95d1c2e174d3b8c4f2e18aaf4a380/roles/60dc7e16aee746d5a3e8aa27d7d8525c => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:39:24.040043 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1d6641-3823-462c-9f96-524be4e21c56 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:24.040355 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1d6641-3823-462c-9f96-524be4e21c56 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:24.040635 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1d6641-3823-462c-9f96-524be4e21c56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:24.071627 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-6d1d6641-3823-462c-9f96-524be4e21c56 None None] MFA Rules not processed for user `a2a95d1c2e174d3b8c4f2e18aaf4a380`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:39:24.072870 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-6d1d6641-3823-462c-9f96-524be4e21c56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:39:24.146990 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1313/2600] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:39:24 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:39:24.151725 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461fada2-5b8c-493c-a5fb-fae34af5b304 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:24.225724 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461fada2-5b8c-493c-a5fb-fae34af5b304 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:24.233519 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a354b9-cccd-411e-b293-c840aa114fab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:24.234444 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a354b9-cccd-411e-b293-c840aa114fab admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:24.234963 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a354b9-cccd-411e-b293-c840aa114fab admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:24.235245 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a354b9-cccd-411e-b293-c840aa114fab admin admin] PATH_INFO: `/v3/projects` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:24.235896 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a354b9-cccd-411e-b293-c840aa114fab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:24.237872 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a354b9-cccd-411e-b293-c840aa114fab admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:24.310576 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1288/2601] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 24 17:39:24 2022] POST /identity/v3/projects => generated 379 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:39:24.318619 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1d6641-3823-462c-9f96-524be4e21c56 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:24.332428 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1d6641-3823-462c-9f96-524be4e21c56 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:24.387889 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:39:24.399617 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaf2e91-c22c-4464-b83d-ace76ac29d64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:24.400401 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaf2e91-c22c-4464-b83d-ace76ac29d64 admin admin] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:24.400686 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaf2e91-c22c-4464-b83d-ace76ac29d64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:24.400956 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaf2e91-c22c-4464-b83d-ace76ac29d64 admin admin] PATH_INFO: `/v3/users` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:24.401728 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eaf2e91-c22c-4464-b83d-ace76ac29d64 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:24.404212 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eaf2e91-c22c-4464-b83d-ace76ac29d64 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:24.479562 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1314/2602] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 24 17:39:24 2022] POST /identity/v3/users => generated 296 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:39:24.484401 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a354b9-cccd-411e-b293-c840aa114fab admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:24.495649 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a354b9-cccd-411e-b293-c840aa114fab admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:24.500665 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90782f9-30d5-402a-85b5-283140d44fde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:24.501441 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90782f9-30d5-402a-85b5-283140d44fde admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:24.501793 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90782f9-30d5-402a-85b5-283140d44fde admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:24.502132 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90782f9-30d5-402a-85b5-283140d44fde admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:24.502930 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90782f9-30d5-402a-85b5-283140d44fde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:24.504620 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90782f9-30d5-402a-85b5-283140d44fde admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:24.515953 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1289/2603] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:39:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:39:24.520315 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaf2e91-c22c-4464-b83d-ace76ac29d64 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:24.532042 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaf2e91-c22c-4464-b83d-ace76ac29d64 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:24.537211 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f28716-d863-4278-9d6d-99d00b618a74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:24.537689 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f28716-d863-4278-9d6d-99d00b618a74 admin admin] REQUEST_METHOD: `PUT` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:24.537971 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f28716-d863-4278-9d6d-99d00b618a74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:24.538259 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f28716-d863-4278-9d6d-99d00b618a74 admin admin] PATH_INFO: `/v3/projects/7b0ed04e0b1c40a1b9c6aae487430d7a/users/01221c0812d84ed7b0cbe62e768f256f/roles/c5346bc54abf445f9900edf94264c850` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:24.563391 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f28716-d863-4278-9d6d-99d00b618a74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b0ed04e0b1c40a1b9c6aae487430d7a, user_id=01221c0812d84ed7b0cbe62e768f256f, role_id=c5346bc54abf445f9900edf94264c850)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:24.566115 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f28716-d863-4278-9d6d-99d00b618a74 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:24.690854 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1315/2604] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 24 17:39:24 2022] PUT /identity/v3/projects/7b0ed04e0b1c40a1b9c6aae487430d7a/users/01221c0812d84ed7b0cbe62e768f256f/roles/c5346bc54abf445f9900edf94264c850 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:39:24.694865 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90782f9-30d5-402a-85b5-283140d44fde admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:24.706230 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90782f9-30d5-402a-85b5-283140d44fde admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:24.737302 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:39:24.766641 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1664ae-9a6a-4a0b-b8c5-d7b8c52e7cca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:24.767429 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1664ae-9a6a-4a0b-b8c5-d7b8c52e7cca admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:24.767788 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1664ae-9a6a-4a0b-b8c5-d7b8c52e7cca admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:24.768145 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1664ae-9a6a-4a0b-b8c5-d7b8c52e7cca admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:24.768890 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1664ae-9a6a-4a0b-b8c5-d7b8c52e7cca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:24.770734 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1664ae-9a6a-4a0b-b8c5-d7b8c52e7cca admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:24.787020 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1290/2605] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:39:24 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:39:24.788845 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f28716-d863-4278-9d6d-99d00b618a74 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:24.801563 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f28716-d863-4278-9d6d-99d00b618a74 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:24.808889 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ca0fc1-9bbd-4043-a73f-ab7eb07b1cb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:24.810388 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ca0fc1-9bbd-4043-a73f-ab7eb07b1cb2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:24.810804 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ca0fc1-9bbd-4043-a73f-ab7eb07b1cb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:24.811169 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ca0fc1-9bbd-4043-a73f-ab7eb07b1cb2 admin admin] PATH_INFO: `/v3/projects/7b0ed04e0b1c40a1b9c6aae487430d7a/users/01221c0812d84ed7b0cbe62e768f256f/roles/c5346bc54abf445f9900edf94264c850` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:24.814538 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ca0fc1-9bbd-4043-a73f-ab7eb07b1cb2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b0ed04e0b1c40a1b9c6aae487430d7a, user_id=01221c0812d84ed7b0cbe62e768f256f, role_id=c5346bc54abf445f9900edf94264c850)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:24.817702 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ca0fc1-9bbd-4043-a73f-ab7eb07b1cb2 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:24.847014 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1316/2606] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 24 17:39:24 2022] PUT /identity/v3/projects/7b0ed04e0b1c40a1b9c6aae487430d7a/users/01221c0812d84ed7b0cbe62e768f256f/roles/c5346bc54abf445f9900edf94264c850 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:39:24.859457 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a58d94f-dff5-4106-816f-7f970bd061b9 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:24.859946 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a58d94f-dff5-4106-816f-7f970bd061b9 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:24.860374 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a58d94f-dff5-4106-816f-7f970bd061b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:24.879850 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ca0fc1-9bbd-4043-a73f-ab7eb07b1cb2 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:24.881621 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-16ca0fc1-9bbd-4043-a73f-ab7eb07b1cb2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:39:24.895543 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-9a58d94f-dff5-4106-816f-7f970bd061b9 None None] MFA Rules not processed for user `01221c0812d84ed7b0cbe62e768f256f`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:39:24.896887 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-9a58d94f-dff5-4106-816f-7f970bd061b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:39:24.955347 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ca0fc1-9bbd-4043-a73f-ab7eb07b1cb2 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:24.959697 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea80ce4-742c-45a4-8f08-f9e45e1da2ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e9ed4d47aef4140bfb2d3bb798cc559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:24.960211 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea80ce4-742c-45a4-8f08-f9e45e1da2ce service neutron] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:24.960531 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea80ce4-742c-45a4-8f08-f9e45e1da2ce service neutron] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:24.960830 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea80ce4-742c-45a4-8f08-f9e45e1da2ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:24.962677 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-9ea80ce4-742c-45a4-8f08-f9e45e1da2ce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:39:24.987181 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1291/2607] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:39:24 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:39:25.000229 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a58d94f-dff5-4106-816f-7f970bd061b9 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:25.001751 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-9a58d94f-dff5-4106-816f-7f970bd061b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:39:25.037663 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea80ce4-742c-45a4-8f08-f9e45e1da2ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:25.039756 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea80ce4-742c-45a4-8f08-f9e45e1da2ce service neutron] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:25.050662 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1317/2608] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:39:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:39:25.068799 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a58d94f-dff5-4106-816f-7f970bd061b9 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:25.078274 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f456fb66-8105-4e5f-99bc-a988c187b787 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be06246d47584b529150ceb98a5cfd95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:25.078924 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f456fb66-8105-4e5f-99bc-a988c187b787 service cinder] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:25.079228 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f456fb66-8105-4e5f-99bc-a988c187b787 service cinder] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:25.079503 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f456fb66-8105-4e5f-99bc-a988c187b787 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:25.089408 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f456fb66-8105-4e5f-99bc-a988c187b787 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:25.091479 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f456fb66-8105-4e5f-99bc-a988c187b787 service cinder] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:25.102205 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1292/2609] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:39:24 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:39:25.203133 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea80ce4-742c-45a4-8f08-f9e45e1da2ce service neutron] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:25.204502 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-9ea80ce4-742c-45a4-8f08-f9e45e1da2ce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:39:25.217079 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea80ce4-742c-45a4-8f08-f9e45e1da2ce service neutron] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:25.220800 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1db7bc4-fee2-4f35-8d1b-57133960f3ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be06246d47584b529150ceb98a5cfd95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:25.221418 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1db7bc4-fee2-4f35-8d1b-57133960f3ec service cinder] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:25.221695 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1db7bc4-fee2-4f35-8d1b-57133960f3ec service cinder] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:25.222014 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1db7bc4-fee2-4f35-8d1b-57133960f3ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:25.300720 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:39:25.311746 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1db7bc4-fee2-4f35-8d1b-57133960f3ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:25.315643 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1db7bc4-fee2-4f35-8d1b-57133960f3ec service cinder] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:25.324279 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1318/2610] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:39:25 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:39:26.612531 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f456fb66-8105-4e5f-99bc-a988c187b787 service cinder] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:26.615330 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-f456fb66-8105-4e5f-99bc-a988c187b787 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:39:26.632977 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f456fb66-8105-4e5f-99bc-a988c187b787 service cinder] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:26.638775 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3c1877-bd95-4da0-9980-d8c1dd954bd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:26.639550 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3c1877-bd95-4da0-9980-d8c1dd954bd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:26.640142 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3c1877-bd95-4da0-9980-d8c1dd954bd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:26.640714 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3c1877-bd95-4da0-9980-d8c1dd954bd7 admin admin] PATH_INFO: `/v3/users/c379c845fccd43c08e8699976d506801` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:26.642456 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3c1877-bd95-4da0-9980-d8c1dd954bd7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c379c845fccd43c08e8699976d506801)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:26.645257 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3c1877-bd95-4da0-9980-d8c1dd954bd7 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:26.912411 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-bb3c1877-bd95-4da0-9980-d8c1dd954bd7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c379c845fccd43c08e8699976d506801'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:26.916555 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:39:27.004272 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-bb3c1877-bd95-4da0-9980-d8c1dd954bd7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c379c845fccd43c08e8699976d506801'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:27.005043 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-bb3c1877-bd95-4da0-9980-d8c1dd954bd7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c379c845fccd43c08e8699976d506801'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:27.015441 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-bb3c1877-bd95-4da0-9980-d8c1dd954bd7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c379c845fccd43c08e8699976d506801'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:27.016775 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-bb3c1877-bd95-4da0-9980-d8c1dd954bd7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c379c845fccd43c08e8699976d506801'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:27.035863 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1293/2611] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 24 17:39:26 2022] DELETE /identity/v3/users/c379c845fccd43c08e8699976d506801 => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:39:27.294642 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1db7bc4-fee2-4f35-8d1b-57133960f3ec service cinder] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:27.312486 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1db7bc4-fee2-4f35-8d1b-57133960f3ec service cinder] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:27.400731 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b973b27-04c9-4dae-883f-b6972432ac5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:27.401941 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b973b27-04c9-4dae-883f-b6972432ac5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:27.402943 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b973b27-04c9-4dae-883f-b6972432ac5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:27.403475 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b973b27-04c9-4dae-883f-b6972432ac5a admin admin] PATH_INFO: `/v3/projects/15cff6e1349f4bb1a2905e6e5231fc2d` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:27.405552 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b973b27-04c9-4dae-883f-b6972432ac5a admin admin] RBAC: Authorizing `identity:delete_project(project_id=15cff6e1349f4bb1a2905e6e5231fc2d)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:27.408240 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b973b27-04c9-4dae-883f-b6972432ac5a admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:27.606690 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-1b973b27-04c9-4dae-883f-b6972432ac5a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '15cff6e1349f4bb1a2905e6e5231fc2d', 'request_id': 'req-1b973b27-04c9-4dae-883f-b6972432ac5a'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:27.672891 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-1b973b27-04c9-4dae-883f-b6972432ac5a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '15cff6e1349f4bb1a2905e6e5231fc2d', 'request_id': 'req-1b973b27-04c9-4dae-883f-b6972432ac5a'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:27.683671 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-1b973b27-04c9-4dae-883f-b6972432ac5a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '15cff6e1349f4bb1a2905e6e5231fc2d', 'request_id': 'req-1b973b27-04c9-4dae-883f-b6972432ac5a'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:27.701809 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-1b973b27-04c9-4dae-883f-b6972432ac5a admin admin] The token cache is being invalidate because project 15cff6e1349f4bb1a2905e6e5231fc2d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81731) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:39:27.702231 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-1b973b27-04c9-4dae-883f-b6972432ac5a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:27.703137 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-1b973b27-04c9-4dae-883f-b6972432ac5a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:27.706970 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1319/2612] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 24 17:39:27 2022] DELETE /identity/v3/projects/15cff6e1349f4bb1a2905e6e5231fc2d => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:39:27.710071 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33125daf-e529-4a42-a5f7-c3be98fef66d None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:27.710562 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33125daf-e529-4a42-a5f7-c3be98fef66d None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:27.710668 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33125daf-e529-4a42-a5f7-c3be98fef66d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:27.744575 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-33125daf-e529-4a42-a5f7-c3be98fef66d None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:39:27.745964 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-33125daf-e529-4a42-a5f7-c3be98fef66d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:39:27.831123 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1294/2613] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:39:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:39:27.839107 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeebf0c-f0a5-4aa3-90d9-3e5734383b41 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:27.839522 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeebf0c-f0a5-4aa3-90d9-3e5734383b41 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:27.839875 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeebf0c-f0a5-4aa3-90d9-3e5734383b41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:27.864919 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-5eeebf0c-f0a5-4aa3-90d9-3e5734383b41 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:39:27.870740 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-5eeebf0c-f0a5-4aa3-90d9-3e5734383b41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:39:27.880281 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1320/2614] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:39:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:39:27.884768 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33125daf-e529-4a42-a5f7-c3be98fef66d None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:27.894987 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33125daf-e529-4a42-a5f7-c3be98fef66d None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:27.900071 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f5fb4e-b413-4dc4-a37c-0704f04c9e31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:27.900687 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f5fb4e-b413-4dc4-a37c-0704f04c9e31 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:27.905307 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f5fb4e-b413-4dc4-a37c-0704f04c9e31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:27.905307 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f5fb4e-b413-4dc4-a37c-0704f04c9e31 admin admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:27.905307 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f5fb4e-b413-4dc4-a37c-0704f04c9e31 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:39:27.905307 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f5fb4e-b413-4dc4-a37c-0704f04c9e31 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:27.905307 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f5fb4e-b413-4dc4-a37c-0704f04c9e31 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:27.921038 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1295/2615] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 24 17:39:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:39:27.922464 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eeebf0c-f0a5-4aa3-90d9-3e5734383b41 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:27.932103 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eeebf0c-f0a5-4aa3-90d9-3e5734383b41 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:27.937543 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c5e456-8735-4e98-acd3-2188c873d64b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:27.939832 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c5e456-8735-4e98-acd3-2188c873d64b admin admin] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:27.939832 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c5e456-8735-4e98-acd3-2188c873d64b admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:27.939832 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c5e456-8735-4e98-acd3-2188c873d64b admin admin] PATH_INFO: `/v3/projects` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:27.939832 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c5e456-8735-4e98-acd3-2188c873d64b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:27.941479 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c5e456-8735-4e98-acd3-2188c873d64b admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:28.066678 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1321/2616] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 24 17:39:27 2022] POST /identity/v3/projects => generated 383 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:39:28.068491 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f5fb4e-b413-4dc4-a37c-0704f04c9e31 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:28.079502 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f5fb4e-b413-4dc4-a37c-0704f04c9e31 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:28.150161 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2659932c-95ef-4158-917c-d275b56faec1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:28.151672 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2659932c-95ef-4158-917c-d275b56faec1 admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:28.152196 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2659932c-95ef-4158-917c-d275b56faec1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:28.152483 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2659932c-95ef-4158-917c-d275b56faec1 admin admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:28.153117 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2659932c-95ef-4158-917c-d275b56faec1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:28.155910 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2659932c-95ef-4158-917c-d275b56faec1 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:28.319252 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1296/2617] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 24 17:39:28 2022] POST /identity/v3/users => generated 292 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:39:28.323435 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c5e456-8735-4e98-acd3-2188c873d64b admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:28.334166 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c5e456-8735-4e98-acd3-2188c873d64b admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:28.338700 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1eb7bb3-a72a-4b9a-83d8-51ce23cdd2f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:28.339383 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eb7bb3-a72a-4b9a-83d8-51ce23cdd2f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:28.339658 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eb7bb3-a72a-4b9a-83d8-51ce23cdd2f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:28.339927 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eb7bb3-a72a-4b9a-83d8-51ce23cdd2f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:28.340660 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1eb7bb3-a72a-4b9a-83d8-51ce23cdd2f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:28.344107 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1eb7bb3-a72a-4b9a-83d8-51ce23cdd2f2 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:28.360024 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1322/2618] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:39:28 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:39:28.361245 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2659932c-95ef-4158-917c-d275b56faec1 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:28.372498 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2659932c-95ef-4158-917c-d275b56faec1 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:28.377318 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9482e34a-0133-43ca-8532-89e482a37cd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:28.377943 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9482e34a-0133-43ca-8532-89e482a37cd7 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:28.378232 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9482e34a-0133-43ca-8532-89e482a37cd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:28.378509 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9482e34a-0133-43ca-8532-89e482a37cd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:28.379190 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9482e34a-0133-43ca-8532-89e482a37cd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:28.380792 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9482e34a-0133-43ca-8532-89e482a37cd7 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:28.392214 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1297/2619] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:39:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:39:28.396591 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1eb7bb3-a72a-4b9a-83d8-51ce23cdd2f2 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:28.407610 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1eb7bb3-a72a-4b9a-83d8-51ce23cdd2f2 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:28.414153 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e941b7-a9a0-4711-a5b7-597271666167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:28.414827 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e941b7-a9a0-4711-a5b7-597271666167 admin admin] REQUEST_METHOD: `PUT` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:28.415183 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e941b7-a9a0-4711-a5b7-597271666167 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:28.415414 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e941b7-a9a0-4711-a5b7-597271666167 admin admin] PATH_INFO: `/v3/projects/727788f6028948f89d3ab8338e15ba09/users/804436148e164d46a65373a1f1c9ed58/roles/60dc7e16aee746d5a3e8aa27d7d8525c` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:28.431231 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e941b7-a9a0-4711-a5b7-597271666167 admin admin] RBAC: Authorizing `identity:create_grant(project_id=727788f6028948f89d3ab8338e15ba09, user_id=804436148e164d46a65373a1f1c9ed58, role_id=60dc7e16aee746d5a3e8aa27d7d8525c)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:28.433782 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e941b7-a9a0-4711-a5b7-597271666167 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:28.535213 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1323/2620] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 24 17:39:28 2022] PUT /identity/v3/projects/727788f6028948f89d3ab8338e15ba09/users/804436148e164d46a65373a1f1c9ed58/roles/60dc7e16aee746d5a3e8aa27d7d8525c => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:39:28.539573 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6f5562-6c5b-4631-b456-f3d1fd8b9f7a None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:28.539573 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6f5562-6c5b-4631-b456-f3d1fd8b9f7a None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:28.539935 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6f5562-6c5b-4631-b456-f3d1fd8b9f7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:28.572266 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-8a6f5562-6c5b-4631-b456-f3d1fd8b9f7a None None] MFA Rules not processed for user `804436148e164d46a65373a1f1c9ed58`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:39:28.573514 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-8a6f5562-6c5b-4631-b456-f3d1fd8b9f7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:39:28.601784 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:39:28.650598 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1298/2621] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:39:28 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:39:28.663533 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e941b7-a9a0-4711-a5b7-597271666167 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:28.665404 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-a4e941b7-a9a0-4711-a5b7-597271666167 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:39:28.688840 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:39:28.770661 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e941b7-a9a0-4711-a5b7-597271666167 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:28.779252 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6226294c-587f-4982-b0c3-6fb345db40be service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557aca9255cb4bbb8af52e796c5c5177', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:28.784282 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6226294c-587f-4982-b0c3-6fb345db40be service nova] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:28.784282 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6226294c-587f-4982-b0c3-6fb345db40be service nova] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:28.784282 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6226294c-587f-4982-b0c3-6fb345db40be service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:28.798394 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6226294c-587f-4982-b0c3-6fb345db40be service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:28.800168 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6226294c-587f-4982-b0c3-6fb345db40be service nova] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:28.812681 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1324/2622] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:39:28 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:39:29.776744 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6f5562-6c5b-4631-b456-f3d1fd8b9f7a None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:29.780605 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-8a6f5562-6c5b-4631-b456-f3d1fd8b9f7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:39:29.891222 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6f5562-6c5b-4631-b456-f3d1fd8b9f7a None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:29.897141 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78800222-08d3-47b6-82c4-79cd48e105b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:29.897931 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78800222-08d3-47b6-82c4-79cd48e105b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:29.898133 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78800222-08d3-47b6-82c4-79cd48e105b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:29.898412 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78800222-08d3-47b6-82c4-79cd48e105b6 admin admin] PATH_INFO: `/v3/users/3c094b49a6df4b72a9b9321f7fae2031` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:29.899684 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78800222-08d3-47b6-82c4-79cd48e105b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c094b49a6df4b72a9b9321f7fae2031)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:29.901637 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78800222-08d3-47b6-82c4-79cd48e105b6 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:30.076153 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-78800222-08d3-47b6-82c4-79cd48e105b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c094b49a6df4b72a9b9321f7fae2031'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:30.124851 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-78800222-08d3-47b6-82c4-79cd48e105b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c094b49a6df4b72a9b9321f7fae2031'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:30.125630 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-78800222-08d3-47b6-82c4-79cd48e105b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c094b49a6df4b72a9b9321f7fae2031'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:30.133990 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-78800222-08d3-47b6-82c4-79cd48e105b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c094b49a6df4b72a9b9321f7fae2031'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:30.134760 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-78800222-08d3-47b6-82c4-79cd48e105b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c094b49a6df4b72a9b9321f7fae2031'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:30.165369 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1299/2623] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 24 17:39:29 2022] DELETE /identity/v3/users/3c094b49a6df4b72a9b9321f7fae2031 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:39:30.169171 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7019ced3-efe8-4eb5-b3f5-da5c4fe4ceba None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:30.169735 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7019ced3-efe8-4eb5-b3f5-da5c4fe4ceba None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:30.169735 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7019ced3-efe8-4eb5-b3f5-da5c4fe4ceba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:30.198174 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-7019ced3-efe8-4eb5-b3f5-da5c4fe4ceba None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:39:30.201388 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-7019ced3-efe8-4eb5-b3f5-da5c4fe4ceba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:39:30.282101 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1325/2624] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:39:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:39:30.296207 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78800222-08d3-47b6-82c4-79cd48e105b6 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:30.297653 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-78800222-08d3-47b6-82c4-79cd48e105b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:39:30.349748 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:39:30.361922 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78800222-08d3-47b6-82c4-79cd48e105b6 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:30.366173 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f3f8c0-adcd-4af7-a12a-14ee4077198b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e9ed4d47aef4140bfb2d3bb798cc559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:30.367252 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f3f8c0-adcd-4af7-a12a-14ee4077198b service neutron] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:30.367795 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f3f8c0-adcd-4af7-a12a-14ee4077198b service neutron] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:30.368282 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f3f8c0-adcd-4af7-a12a-14ee4077198b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:30.383755 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f3f8c0-adcd-4af7-a12a-14ee4077198b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:30.385271 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f3f8c0-adcd-4af7-a12a-14ee4077198b service neutron] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:30.397198 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1300/2625] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:39:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:39:30.991174 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7019ced3-efe8-4eb5-b3f5-da5c4fe4ceba None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:30.992923 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-7019ced3-efe8-4eb5-b3f5-da5c4fe4ceba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:39:31.011765 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7019ced3-efe8-4eb5-b3f5-da5c4fe4ceba None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:31.024501 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab46d9f-4c52-4919-a42f-066a9681aea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:31.025488 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab46d9f-4c52-4919-a42f-066a9681aea5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:31.026075 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab46d9f-4c52-4919-a42f-066a9681aea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:31.026388 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab46d9f-4c52-4919-a42f-066a9681aea5 admin admin] PATH_INFO: `/v3/projects/1a43e26116624ddc98cc48f4e799c9bb` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:31.028706 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab46d9f-4c52-4919-a42f-066a9681aea5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a43e26116624ddc98cc48f4e799c9bb)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:31.030565 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab46d9f-4c52-4919-a42f-066a9681aea5 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:31.232141 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-bab46d9f-4c52-4919-a42f-066a9681aea5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a43e26116624ddc98cc48f4e799c9bb', 'request_id': 'req-bab46d9f-4c52-4919-a42f-066a9681aea5'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:31.318358 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-bab46d9f-4c52-4919-a42f-066a9681aea5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a43e26116624ddc98cc48f4e799c9bb', 'request_id': 'req-bab46d9f-4c52-4919-a42f-066a9681aea5'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:31.325567 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-bab46d9f-4c52-4919-a42f-066a9681aea5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a43e26116624ddc98cc48f4e799c9bb', 'request_id': 'req-bab46d9f-4c52-4919-a42f-066a9681aea5'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:31.344228 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-bab46d9f-4c52-4919-a42f-066a9681aea5 admin admin] The token cache is being invalidate because project 1a43e26116624ddc98cc48f4e799c9bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81731) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:39:31.344519 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-bab46d9f-4c52-4919-a42f-066a9681aea5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:31.346033 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-bab46d9f-4c52-4919-a42f-066a9681aea5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:31.347801 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1326/2626] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 24 17:39:30 2022] DELETE /identity/v3/projects/1a43e26116624ddc98cc48f4e799c9bb => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:39:31.351839 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f3f8c0-adcd-4af7-a12a-14ee4077198b service neutron] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:31.363148 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f3f8c0-adcd-4af7-a12a-14ee4077198b service neutron] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:31.430707 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a14f3a9-71af-44c5-9e71-b5c783a6c0d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:31.431519 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a14f3a9-71af-44c5-9e71-b5c783a6c0d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:31.431919 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a14f3a9-71af-44c5-9e71-b5c783a6c0d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:31.432251 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a14f3a9-71af-44c5-9e71-b5c783a6c0d2 admin admin] PATH_INFO: `/v3/users/a2a95d1c2e174d3b8c4f2e18aaf4a380` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:31.433488 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a14f3a9-71af-44c5-9e71-b5c783a6c0d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2a95d1c2e174d3b8c4f2e18aaf4a380)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:31.435479 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a14f3a9-71af-44c5-9e71-b5c783a6c0d2 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:31.686430 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-9a14f3a9-71af-44c5-9e71-b5c783a6c0d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2a95d1c2e174d3b8c4f2e18aaf4a380'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:31.749900 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-9a14f3a9-71af-44c5-9e71-b5c783a6c0d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2a95d1c2e174d3b8c4f2e18aaf4a380'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:31.750674 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-9a14f3a9-71af-44c5-9e71-b5c783a6c0d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2a95d1c2e174d3b8c4f2e18aaf4a380'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:31.758953 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-9a14f3a9-71af-44c5-9e71-b5c783a6c0d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2a95d1c2e174d3b8c4f2e18aaf4a380'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:31.759685 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-9a14f3a9-71af-44c5-9e71-b5c783a6c0d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2a95d1c2e174d3b8c4f2e18aaf4a380'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:31.787534 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1301/2627] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 24 17:39:31 2022] DELETE /identity/v3/users/a2a95d1c2e174d3b8c4f2e18aaf4a380 => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:39:32.439193 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab46d9f-4c52-4919-a42f-066a9681aea5 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:32.442079 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-bab46d9f-4c52-4919-a42f-066a9681aea5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:39:32.456561 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab46d9f-4c52-4919-a42f-066a9681aea5 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:32.509113 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:39:32.551373 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d53b76d-89cc-4279-a325-2702f3612e52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:32.551925 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d53b76d-89cc-4279-a325-2702f3612e52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:32.552370 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d53b76d-89cc-4279-a325-2702f3612e52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:32.552845 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d53b76d-89cc-4279-a325-2702f3612e52 admin admin] PATH_INFO: `/v3/projects/aca526fc4af5431581ac05a15d035a9d` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:32.554379 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d53b76d-89cc-4279-a325-2702f3612e52 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aca526fc4af5431581ac05a15d035a9d)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:32.556578 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d53b76d-89cc-4279-a325-2702f3612e52 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:32.694047 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-3d53b76d-89cc-4279-a325-2702f3612e52 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aca526fc4af5431581ac05a15d035a9d', 'request_id': 'req-3d53b76d-89cc-4279-a325-2702f3612e52'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:32.741678 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-3d53b76d-89cc-4279-a325-2702f3612e52 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aca526fc4af5431581ac05a15d035a9d', 'request_id': 'req-3d53b76d-89cc-4279-a325-2702f3612e52'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:32.755186 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-3d53b76d-89cc-4279-a325-2702f3612e52 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aca526fc4af5431581ac05a15d035a9d', 'request_id': 'req-3d53b76d-89cc-4279-a325-2702f3612e52'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:32.769756 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-3d53b76d-89cc-4279-a325-2702f3612e52 admin admin] The token cache is being invalidate because project aca526fc4af5431581ac05a15d035a9d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81731) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:39:32.770133 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-3d53b76d-89cc-4279-a325-2702f3612e52 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:32.771097 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-3d53b76d-89cc-4279-a325-2702f3612e52 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:32.772847 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1327/2628] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 24 17:39:32 2022] DELETE /identity/v3/projects/aca526fc4af5431581ac05a15d035a9d => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:39:32.777137 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a14f3a9-71af-44c5-9e71-b5c783a6c0d2 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:32.779247 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-9a14f3a9-71af-44c5-9e71-b5c783a6c0d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:39:32.796869 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a14f3a9-71af-44c5-9e71-b5c783a6c0d2 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:32.863126 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b08a1f-a294-44b3-a2ef-004f707f26f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:32.863882 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b08a1f-a294-44b3-a2ef-004f707f26f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:32.864269 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b08a1f-a294-44b3-a2ef-004f707f26f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:32.864612 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b08a1f-a294-44b3-a2ef-004f707f26f6 admin admin] PATH_INFO: `/v3/users/01221c0812d84ed7b0cbe62e768f256f` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:32.865878 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b08a1f-a294-44b3-a2ef-004f707f26f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=01221c0812d84ed7b0cbe62e768f256f)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:32.867849 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b08a1f-a294-44b3-a2ef-004f707f26f6 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:32.874461 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:39:33.081209 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-47b08a1f-a294-44b3-a2ef-004f707f26f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01221c0812d84ed7b0cbe62e768f256f'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:33.157970 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-47b08a1f-a294-44b3-a2ef-004f707f26f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01221c0812d84ed7b0cbe62e768f256f'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:33.158650 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-47b08a1f-a294-44b3-a2ef-004f707f26f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01221c0812d84ed7b0cbe62e768f256f'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:33.167992 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-47b08a1f-a294-44b3-a2ef-004f707f26f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01221c0812d84ed7b0cbe62e768f256f'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:33.168868 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-47b08a1f-a294-44b3-a2ef-004f707f26f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01221c0812d84ed7b0cbe62e768f256f'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:33.207200 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1302/2629] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 24 17:39:32 2022] DELETE /identity/v3/users/01221c0812d84ed7b0cbe62e768f256f => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:39:33.750313 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d53b76d-89cc-4279-a325-2702f3612e52 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:33.752536 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-3d53b76d-89cc-4279-a325-2702f3612e52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:39:33.770353 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d53b76d-89cc-4279-a325-2702f3612e52 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:33.861392 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce08218a-bb7c-4676-a131-8631ace7b276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:33.862309 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce08218a-bb7c-4676-a131-8631ace7b276 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:33.862737 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce08218a-bb7c-4676-a131-8631ace7b276 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:33.863335 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce08218a-bb7c-4676-a131-8631ace7b276 admin admin] PATH_INFO: `/v3/projects/7b0ed04e0b1c40a1b9c6aae487430d7a` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:33.865004 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce08218a-bb7c-4676-a131-8631ace7b276 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b0ed04e0b1c40a1b9c6aae487430d7a)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:33.870586 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce08218a-bb7c-4676-a131-8631ace7b276 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:33.996870 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-ce08218a-bb7c-4676-a131-8631ace7b276 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b0ed04e0b1c40a1b9c6aae487430d7a', 'request_id': 'req-ce08218a-bb7c-4676-a131-8631ace7b276'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:34.051488 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b08a1f-a294-44b3-a2ef-004f707f26f6 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:34.053031 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-47b08a1f-a294-44b3-a2ef-004f707f26f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:39:34.055965 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-ce08218a-bb7c-4676-a131-8631ace7b276 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b0ed04e0b1c40a1b9c6aae487430d7a', 'request_id': 'req-ce08218a-bb7c-4676-a131-8631ace7b276'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:34.058885 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:39:34.067778 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-ce08218a-bb7c-4676-a131-8631ace7b276 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b0ed04e0b1c40a1b9c6aae487430d7a', 'request_id': 'req-ce08218a-bb7c-4676-a131-8631ace7b276'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:34.070218 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b08a1f-a294-44b3-a2ef-004f707f26f6 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:34.078317 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-ce08218a-bb7c-4676-a131-8631ace7b276 admin admin] The token cache is being invalidate because project 7b0ed04e0b1c40a1b9c6aae487430d7a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81731) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:39:34.078608 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-ce08218a-bb7c-4676-a131-8631ace7b276 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:34.079312 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-ce08218a-bb7c-4676-a131-8631ace7b276 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:34.084026 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1328/2630] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 24 17:39:33 2022] DELETE /identity/v3/projects/7b0ed04e0b1c40a1b9c6aae487430d7a => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:39:34.087073 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7085f378-d229-46eb-a70d-a0b3892eb355 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:34.087370 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7085f378-d229-46eb-a70d-a0b3892eb355 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:34.087650 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7085f378-d229-46eb-a70d-a0b3892eb355 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:34.109006 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-7085f378-d229-46eb-a70d-a0b3892eb355 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:39:34.110253 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-7085f378-d229-46eb-a70d-a0b3892eb355 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:39:34.156423 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1594240b-1d66-49fc-bfa2-4d1632160b80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:34.157735 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1594240b-1d66-49fc-bfa2-4d1632160b80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:34.158169 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1594240b-1d66-49fc-bfa2-4d1632160b80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:34.158281 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1594240b-1d66-49fc-bfa2-4d1632160b80 admin admin] PATH_INFO: `/v3/users/47da700d7d0f4d5cb1ea984a82d643bf` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:34.159538 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1594240b-1d66-49fc-bfa2-4d1632160b80 admin admin] RBAC: Authorizing `identity:delete_user(user_id=47da700d7d0f4d5cb1ea984a82d643bf)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:34.161485 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1594240b-1d66-49fc-bfa2-4d1632160b80 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:34.185250 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1329/2631] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:39:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:39:34.253136 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9551375-5b4e-4baf-afe6-e1f432feb273 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:34.253424 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9551375-5b4e-4baf-afe6-e1f432feb273 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:34.253706 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9551375-5b4e-4baf-afe6-e1f432feb273 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:34.274930 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-b9551375-5b4e-4baf-afe6-e1f432feb273 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:39:34.276207 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-b9551375-5b4e-4baf-afe6-e1f432feb273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:39:34.284847 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1330/2632] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:39:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:39:34.289264 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9551375-5b4e-4baf-afe6-e1f432feb273 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:34.299203 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9551375-5b4e-4baf-afe6-e1f432feb273 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:34.304068 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde41c26-accc-4d20-852e-aef820bc5869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:34.304691 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde41c26-accc-4d20-852e-aef820bc5869 admin admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:34.305014 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde41c26-accc-4d20-852e-aef820bc5869 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:34.305234 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde41c26-accc-4d20-852e-aef820bc5869 admin admin] PATH_INFO: `/v3/domains` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:34.305896 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde41c26-accc-4d20-852e-aef820bc5869 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:39:34.306199 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde41c26-accc-4d20-852e-aef820bc5869 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:34.307831 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde41c26-accc-4d20-852e-aef820bc5869 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:34.321677 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1331/2633] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 24 17:39:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:39:34.327107 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde41c26-accc-4d20-852e-aef820bc5869 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:34.337253 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde41c26-accc-4d20-852e-aef820bc5869 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:34.342111 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9c1889-3735-4307-a397-2953fc800abb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:34.343460 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9c1889-3735-4307-a397-2953fc800abb admin admin] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:34.343460 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9c1889-3735-4307-a397-2953fc800abb admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:34.343460 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9c1889-3735-4307-a397-2953fc800abb admin admin] PATH_INFO: `/v3/projects` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:34.343921 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9c1889-3735-4307-a397-2953fc800abb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:34.345824 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9c1889-3735-4307-a397-2953fc800abb admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:34.357858 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-1594240b-1d66-49fc-bfa2-4d1632160b80 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47da700d7d0f4d5cb1ea984a82d643bf'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:34.432867 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-1594240b-1d66-49fc-bfa2-4d1632160b80 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47da700d7d0f4d5cb1ea984a82d643bf'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:34.433507 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-1594240b-1d66-49fc-bfa2-4d1632160b80 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47da700d7d0f4d5cb1ea984a82d643bf'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:34.442045 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-1594240b-1d66-49fc-bfa2-4d1632160b80 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47da700d7d0f4d5cb1ea984a82d643bf'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:34.442787 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-1594240b-1d66-49fc-bfa2-4d1632160b80 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47da700d7d0f4d5cb1ea984a82d643bf'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:34.452334 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1332/2634] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 24 17:39:34 2022] POST /identity/v3/projects => generated 383 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:39:34.456709 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9c1889-3735-4307-a397-2953fc800abb admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:34.458409 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-fe9c1889-3735-4307-a397-2953fc800abb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:39:34.463289 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1303/2635] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 24 17:39:34 2022] DELETE /identity/v3/users/47da700d7d0f4d5cb1ea984a82d643bf => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:39:34.533039 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9c1889-3735-4307-a397-2953fc800abb admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:34.598327 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:39:34.609804 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd855c5-964c-4e2c-81fb-a022f8a93053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:34.610790 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd855c5-964c-4e2c-81fb-a022f8a93053 admin admin] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:34.611212 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd855c5-964c-4e2c-81fb-a022f8a93053 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:34.611572 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd855c5-964c-4e2c-81fb-a022f8a93053 admin admin] PATH_INFO: `/v3/users` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:34.612281 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd855c5-964c-4e2c-81fb-a022f8a93053 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:34.614944 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd855c5-964c-4e2c-81fb-a022f8a93053 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:34.725519 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1333/2636] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 24 17:39:34 2022] POST /identity/v3/users => generated 292 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:39:34.730994 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1594240b-1d66-49fc-bfa2-4d1632160b80 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:34.738458 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:39:34.742313 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1594240b-1d66-49fc-bfa2-4d1632160b80 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:34.747197 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dba373-ba25-41e9-8128-54651eead12e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:34.747841 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dba373-ba25-41e9-8128-54651eead12e admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:34.748749 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dba373-ba25-41e9-8128-54651eead12e admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:34.749022 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dba373-ba25-41e9-8128-54651eead12e admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:34.749670 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76dba373-ba25-41e9-8128-54651eead12e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:34.754173 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76dba373-ba25-41e9-8128-54651eead12e admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:34.771566 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1304/2637] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:39:34 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:39:34.772880 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dba373-ba25-41e9-8128-54651eead12e admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:34.773047 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd855c5-964c-4e2c-81fb-a022f8a93053 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:34.775625 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-4fd855c5-964c-4e2c-81fb-a022f8a93053 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:39:34.793274 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dba373-ba25-41e9-8128-54651eead12e admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:34.797176 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd855c5-964c-4e2c-81fb-a022f8a93053 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:34.801595 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2aaa2e2-cbe2-4f71-b252-198257b6713b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:34.802857 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aaa2e2-cbe2-4f71-b252-198257b6713b admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:34.811379 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aaa2e2-cbe2-4f71-b252-198257b6713b admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:34.812192 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67545871-4f90-4b55-a089-cb0850591459 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:34.812319 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aaa2e2-cbe2-4f71-b252-198257b6713b admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:34.812416 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67545871-4f90-4b55-a089-cb0850591459 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:34.812416 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67545871-4f90-4b55-a089-cb0850591459 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:34.812595 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2aaa2e2-cbe2-4f71-b252-198257b6713b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:34.812686 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67545871-4f90-4b55-a089-cb0850591459 admin admin] PATH_INFO: `/v3/projects/036fc4bf065249e29d0e1e90d23bae19` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:34.812686 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67545871-4f90-4b55-a089-cb0850591459 admin admin] RBAC: Authorizing `identity:delete_project(project_id=036fc4bf065249e29d0e1e90d23bae19)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:34.812870 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2aaa2e2-cbe2-4f71-b252-198257b6713b admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:34.812962 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67545871-4f90-4b55-a089-cb0850591459 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:34.833304 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1305/2638] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:39:34 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:39:34.837747 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2aaa2e2-cbe2-4f71-b252-198257b6713b admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:34.854323 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2aaa2e2-cbe2-4f71-b252-198257b6713b admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:34.859596 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fccf3df-dd1e-4eb6-9800-c1bdebb86008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:34.860265 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fccf3df-dd1e-4eb6-9800-c1bdebb86008 admin admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:34.860545 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fccf3df-dd1e-4eb6-9800-c1bdebb86008 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:34.860821 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fccf3df-dd1e-4eb6-9800-c1bdebb86008 admin admin] PATH_INFO: `/v3/projects/509c2c7a42784298957c57f2a114babf/users/036faf6961ac45e08d46ea03257b27ea/roles/60dc7e16aee746d5a3e8aa27d7d8525c` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:34.878176 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fccf3df-dd1e-4eb6-9800-c1bdebb86008 admin admin] RBAC: Authorizing `identity:create_grant(project_id=509c2c7a42784298957c57f2a114babf, user_id=036faf6961ac45e08d46ea03257b27ea, role_id=60dc7e16aee746d5a3e8aa27d7d8525c)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:34.880997 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fccf3df-dd1e-4eb6-9800-c1bdebb86008 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:34.942190 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1306/2639] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 24 17:39:34 2022] PUT /identity/v3/projects/509c2c7a42784298957c57f2a114babf/users/036faf6961ac45e08d46ea03257b27ea/roles/60dc7e16aee746d5a3e8aa27d7d8525c => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:39:34.955091 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-67545871-4f90-4b55-a089-cb0850591459 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '036fc4bf065249e29d0e1e90d23bae19', 'request_id': 'req-67545871-4f90-4b55-a089-cb0850591459'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:34.955672 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69af2085-f2cc-4771-94f6-f6065d81b0b8 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:34.956293 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69af2085-f2cc-4771-94f6-f6065d81b0b8 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:34.956821 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69af2085-f2cc-4771-94f6-f6065d81b0b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:34.987709 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-69af2085-f2cc-4771-94f6-f6065d81b0b8 None None] MFA Rules not processed for user `036faf6961ac45e08d46ea03257b27ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:39:34.989373 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-69af2085-f2cc-4771-94f6-f6065d81b0b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:39:35.009520 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-67545871-4f90-4b55-a089-cb0850591459 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '036fc4bf065249e29d0e1e90d23bae19', 'request_id': 'req-67545871-4f90-4b55-a089-cb0850591459'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:35.019289 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-67545871-4f90-4b55-a089-cb0850591459 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '036fc4bf065249e29d0e1e90d23bae19', 'request_id': 'req-67545871-4f90-4b55-a089-cb0850591459'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:35.034285 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-67545871-4f90-4b55-a089-cb0850591459 admin admin] The token cache is being invalidate because project 036fc4bf065249e29d0e1e90d23bae19 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81731) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:39:35.034711 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-67545871-4f90-4b55-a089-cb0850591459 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:35.035702 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-67545871-4f90-4b55-a089-cb0850591459 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:35.037652 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1334/2640] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 24 17:39:34 2022] DELETE /identity/v3/projects/036fc4bf065249e29d0e1e90d23bae19 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:39:35.041538 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67545871-4f90-4b55-a089-cb0850591459 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:35.042992 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-67545871-4f90-4b55-a089-cb0850591459 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:39:35.065491 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1307/2641] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:39:34 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:39:35.074812 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69af2085-f2cc-4771-94f6-f6065d81b0b8 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:35.077349 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-69af2085-f2cc-4771-94f6-f6065d81b0b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:39:35.131702 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67545871-4f90-4b55-a089-cb0850591459 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:35.140058 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e5dd58-e2c5-48fa-941c-0a84c9dd78d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:35.140878 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e5dd58-e2c5-48fa-941c-0a84c9dd78d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:35.141163 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e5dd58-e2c5-48fa-941c-0a84c9dd78d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:35.141439 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e5dd58-e2c5-48fa-941c-0a84c9dd78d5 admin admin] PATH_INFO: `/v3/users/6a9c759c8eeb46e4b1ef2c1c1e2a7201` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:35.153471 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69af2085-f2cc-4771-94f6-f6065d81b0b8 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:35.157186 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e5dd58-e2c5-48fa-941c-0a84c9dd78d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a9c759c8eeb46e4b1ef2c1c1e2a7201)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:35.159400 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e5dd58-e2c5-48fa-941c-0a84c9dd78d5 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:35.160208 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e6baf3-b3a8-4c82-81d9-799030cde1e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:35.160925 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e6baf3-b3a8-4c82-81d9-799030cde1e8 admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:35.161208 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e6baf3-b3a8-4c82-81d9-799030cde1e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:35.161483 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e6baf3-b3a8-4c82-81d9-799030cde1e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:35.162168 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e6baf3-b3a8-4c82-81d9-799030cde1e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:35.164211 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e6baf3-b3a8-4c82-81d9-799030cde1e8 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:35.251263 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1308/2642] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 24 17:39:35 2022] POST /identity/v3/projects => generated 381 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:39:35.255285 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e6baf3-b3a8-4c82-81d9-799030cde1e8 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:35.290953 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:39:35.322672 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e6baf3-b3a8-4c82-81d9-799030cde1e8 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:35.324316 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a8cefe-674f-44b7-8c98-a0ba1e95b3aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:35.325013 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a8cefe-674f-44b7-8c98-a0ba1e95b3aa admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:35.325294 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a8cefe-674f-44b7-8c98-a0ba1e95b3aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:35.325568 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a8cefe-674f-44b7-8c98-a0ba1e95b3aa admin admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:35.326713 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a8cefe-674f-44b7-8c98-a0ba1e95b3aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:35.328677 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a8cefe-674f-44b7-8c98-a0ba1e95b3aa admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:35.355302 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-51e5dd58-e2c5-48fa-941c-0a84c9dd78d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a9c759c8eeb46e4b1ef2c1c1e2a7201'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:35.362728 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=81,keystone:INSERT=4,keystone:DELETE=15 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:39:35.368024 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-51e5dd58-e2c5-48fa-941c-0a84c9dd78d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a9c759c8eeb46e4b1ef2c1c1e2a7201'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:35.473601 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1309/2643] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 24 17:39:35 2022] POST /identity/v3/users => generated 297 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:39:35.477649 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a8cefe-674f-44b7-8c98-a0ba1e95b3aa admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:35.487578 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a8cefe-674f-44b7-8c98-a0ba1e95b3aa admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:35.493278 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a6f05d-c39b-4741-a84b-35c9ace18829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:35.493874 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a6f05d-c39b-4741-a84b-35c9ace18829 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:35.494154 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a6f05d-c39b-4741-a84b-35c9ace18829 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:35.494426 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a6f05d-c39b-4741-a84b-35c9ace18829 admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:35.495114 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a6f05d-c39b-4741-a84b-35c9ace18829 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:35.496692 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a6f05d-c39b-4741-a84b-35c9ace18829 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:35.506862 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1310/2644] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:39:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:39:35.511362 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a6f05d-c39b-4741-a84b-35c9ace18829 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:35.521642 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-51e5dd58-e2c5-48fa-941c-0a84c9dd78d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a9c759c8eeb46e4b1ef2c1c1e2a7201'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:35.521642 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-51e5dd58-e2c5-48fa-941c-0a84c9dd78d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a9c759c8eeb46e4b1ef2c1c1e2a7201'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:35.523526 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a6f05d-c39b-4741-a84b-35c9ace18829 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:35.523902 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-51e5dd58-e2c5-48fa-941c-0a84c9dd78d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a9c759c8eeb46e4b1ef2c1c1e2a7201'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:35.528422 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8a8dcb-2749-4bd8-91d9-4073962eae4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:35.529119 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8a8dcb-2749-4bd8-91d9-4073962eae4f admin admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:35.530196 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8a8dcb-2749-4bd8-91d9-4073962eae4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:35.530473 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8a8dcb-2749-4bd8-91d9-4073962eae4f admin admin] PATH_INFO: `/v3/projects/25999f14ae2245a5a7b8a6075e31d9f5/users/c8b239a784854858ba1cb01f1f083d8e/roles/c5346bc54abf445f9900edf94264c850` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:35.532976 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1335/2645] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 24 17:39:35 2022] DELETE /identity/v3/users/6a9c759c8eeb46e4b1ef2c1c1e2a7201 => generated 0 bytes in 492 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:39:35.557144 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8a8dcb-2749-4bd8-91d9-4073962eae4f admin admin] RBAC: Authorizing `identity:create_grant(project_id=25999f14ae2245a5a7b8a6075e31d9f5, user_id=c8b239a784854858ba1cb01f1f083d8e, role_id=c5346bc54abf445f9900edf94264c850)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:35.557144 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8a8dcb-2749-4bd8-91d9-4073962eae4f admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:35.633288 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1311/2646] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 24 17:39:35 2022] PUT /identity/v3/projects/25999f14ae2245a5a7b8a6075e31d9f5/users/c8b239a784854858ba1cb01f1f083d8e/roles/c5346bc54abf445f9900edf94264c850 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:39:35.638237 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e5dd58-e2c5-48fa-941c-0a84c9dd78d5 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:35.641031 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-51e5dd58-e2c5-48fa-941c-0a84c9dd78d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:39:35.723290 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e5dd58-e2c5-48fa-941c-0a84c9dd78d5 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:35.788580 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847b1649-b983-4105-918d-482010e41e09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:35.789353 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847b1649-b983-4105-918d-482010e41e09 admin admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:35.789461 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847b1649-b983-4105-918d-482010e41e09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:35.789727 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847b1649-b983-4105-918d-482010e41e09 admin admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:35.790377 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847b1649-b983-4105-918d-482010e41e09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:35.792025 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847b1649-b983-4105-918d-482010e41e09 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:35.807280 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1336/2647] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:39:35 2022] GET /identity/v3/roles => generated 1420 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:39:35.808714 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8a8dcb-2749-4bd8-91d9-4073962eae4f admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:35.820408 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8a8dcb-2749-4bd8-91d9-4073962eae4f admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:35.825341 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577d7709-f17d-44d7-a27c-4e6c09a1745a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:35.826145 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577d7709-f17d-44d7-a27c-4e6c09a1745a admin admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:35.826500 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577d7709-f17d-44d7-a27c-4e6c09a1745a admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:35.826784 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577d7709-f17d-44d7-a27c-4e6c09a1745a admin admin] PATH_INFO: `/v3/projects/25999f14ae2245a5a7b8a6075e31d9f5/users/c8b239a784854858ba1cb01f1f083d8e/roles/c5346bc54abf445f9900edf94264c850` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:35.828845 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577d7709-f17d-44d7-a27c-4e6c09a1745a admin admin] RBAC: Authorizing `identity:create_grant(project_id=25999f14ae2245a5a7b8a6075e31d9f5, user_id=c8b239a784854858ba1cb01f1f083d8e, role_id=c5346bc54abf445f9900edf94264c850)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:35.831530 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577d7709-f17d-44d7-a27c-4e6c09a1745a admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:35.846946 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1312/2648] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 24 17:39:35 2022] PUT /identity/v3/projects/25999f14ae2245a5a7b8a6075e31d9f5/users/c8b239a784854858ba1cb01f1f083d8e/roles/c5346bc54abf445f9900edf94264c850 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:39:35.860652 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebbb828-0d18-4ef4-b9eb-697f2b132046 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:35.860965 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebbb828-0d18-4ef4-b9eb-697f2b132046 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:35.861220 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebbb828-0d18-4ef4-b9eb-697f2b132046 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:35.907889 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-bebbb828-0d18-4ef4-b9eb-697f2b132046 None None] MFA Rules not processed for user `c8b239a784854858ba1cb01f1f083d8e`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:39:35.909126 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-bebbb828-0d18-4ef4-b9eb-697f2b132046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:39:36.023100 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1337/2649] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:39:35 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:39:36.040430 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577d7709-f17d-44d7-a27c-4e6c09a1745a admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:36.042339 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-577d7709-f17d-44d7-a27c-4e6c09a1745a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:39:36.103089 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebbb828-0d18-4ef4-b9eb-697f2b132046 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:36.104516 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-bebbb828-0d18-4ef4-b9eb-697f2b132046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:39:36.109660 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:39:36.110667 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577d7709-f17d-44d7-a27c-4e6c09a1745a admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:36.116648 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82490e03-c2f7-4488-b8b4-4209329f8770 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be06246d47584b529150ceb98a5cfd95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:36.118275 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82490e03-c2f7-4488-b8b4-4209329f8770 service cinder] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:36.118275 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82490e03-c2f7-4488-b8b4-4209329f8770 service cinder] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:36.118275 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82490e03-c2f7-4488-b8b4-4209329f8770 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:36.130716 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82490e03-c2f7-4488-b8b4-4209329f8770 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:36.132217 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82490e03-c2f7-4488-b8b4-4209329f8770 service cinder] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:36.147523 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1313/2650] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:39:36 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:39:36.198213 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebbb828-0d18-4ef4-b9eb-697f2b132046 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:36.203223 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61272b54-b0a6-4f9d-9034-e40401112b13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:36.203912 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61272b54-b0a6-4f9d-9034-e40401112b13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:36.204196 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61272b54-b0a6-4f9d-9034-e40401112b13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:36.204560 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61272b54-b0a6-4f9d-9034-e40401112b13 admin admin] PATH_INFO: `/v3/projects/1120639c2b904bdbbafa3bf6d6343270` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:36.220075 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61272b54-b0a6-4f9d-9034-e40401112b13 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1120639c2b904bdbbafa3bf6d6343270)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:36.222456 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61272b54-b0a6-4f9d-9034-e40401112b13 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:36.350413 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82490e03-c2f7-4488-b8b4-4209329f8770 service cinder] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:36.360478 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82490e03-c2f7-4488-b8b4-4209329f8770 service cinder] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:36.364724 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c27c559-b647-4136-8687-d0bc7a3eb00b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be06246d47584b529150ceb98a5cfd95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:36.365328 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c27c559-b647-4136-8687-d0bc7a3eb00b service cinder] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:36.365607 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c27c559-b647-4136-8687-d0bc7a3eb00b service cinder] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:36.365870 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c27c559-b647-4136-8687-d0bc7a3eb00b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:36.367632 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-4c27c559-b647-4136-8687-d0bc7a3eb00b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:39:36.381909 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-61272b54-b0a6-4f9d-9034-e40401112b13 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1120639c2b904bdbbafa3bf6d6343270', 'request_id': 'req-61272b54-b0a6-4f9d-9034-e40401112b13'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:36.430639 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c27c559-b647-4136-8687-d0bc7a3eb00b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:36.432229 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c27c559-b647-4136-8687-d0bc7a3eb00b service cinder] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:36.441942 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1314/2651] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:39:36 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:39:36.449268 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-61272b54-b0a6-4f9d-9034-e40401112b13 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1120639c2b904bdbbafa3bf6d6343270', 'request_id': 'req-61272b54-b0a6-4f9d-9034-e40401112b13'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:36.456823 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-61272b54-b0a6-4f9d-9034-e40401112b13 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1120639c2b904bdbbafa3bf6d6343270', 'request_id': 'req-61272b54-b0a6-4f9d-9034-e40401112b13'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:36.470847 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-61272b54-b0a6-4f9d-9034-e40401112b13 admin admin] The token cache is being invalidate because project 1120639c2b904bdbbafa3bf6d6343270 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81731) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:39:36.471567 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-61272b54-b0a6-4f9d-9034-e40401112b13 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:36.472677 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-61272b54-b0a6-4f9d-9034-e40401112b13 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:36.475328 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1338/2652] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 24 17:39:36 2022] DELETE /identity/v3/projects/1120639c2b904bdbbafa3bf6d6343270 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:39:37.012231 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c27c559-b647-4136-8687-d0bc7a3eb00b service cinder] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:37.025279 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c27c559-b647-4136-8687-d0bc7a3eb00b service cinder] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:37.039038 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:39:37.098792 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6633528e-b0a1-4f82-858f-c38b791635af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:37.099746 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6633528e-b0a1-4f82-858f-c38b791635af admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:37.100325 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6633528e-b0a1-4f82-858f-c38b791635af admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:37.100694 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6633528e-b0a1-4f82-858f-c38b791635af admin admin] PATH_INFO: `/v3/users/036faf6961ac45e08d46ea03257b27ea` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:37.101965 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6633528e-b0a1-4f82-858f-c38b791635af admin admin] RBAC: Authorizing `identity:delete_user(user_id=036faf6961ac45e08d46ea03257b27ea)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:37.104394 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6633528e-b0a1-4f82-858f-c38b791635af admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:37.413954 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-6633528e-b0a1-4f82-858f-c38b791635af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '036faf6961ac45e08d46ea03257b27ea'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:37.507223 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-6633528e-b0a1-4f82-858f-c38b791635af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '036faf6961ac45e08d46ea03257b27ea'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:37.507557 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-6633528e-b0a1-4f82-858f-c38b791635af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '036faf6961ac45e08d46ea03257b27ea'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:37.516300 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-6633528e-b0a1-4f82-858f-c38b791635af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '036faf6961ac45e08d46ea03257b27ea'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:37.517702 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-6633528e-b0a1-4f82-858f-c38b791635af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '036faf6961ac45e08d46ea03257b27ea'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:37.542334 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1315/2653] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 24 17:39:37 2022] DELETE /identity/v3/users/036faf6961ac45e08d46ea03257b27ea => generated 0 bytes in 531 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:39:37.549830 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1633ed33-2d06-4f1e-bc4f-2f6bda73ec7d None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:37.550438 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1633ed33-2d06-4f1e-bc4f-2f6bda73ec7d None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:37.554960 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1633ed33-2d06-4f1e-bc4f-2f6bda73ec7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:37.573220 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-1633ed33-2d06-4f1e-bc4f-2f6bda73ec7d None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:39:37.574627 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-1633ed33-2d06-4f1e-bc4f-2f6bda73ec7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:39:37.638653 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1339/2654] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:39:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:39:37.650477 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6633528e-b0a1-4f82-858f-c38b791635af admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:37.653051 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-6633528e-b0a1-4f82-858f-c38b791635af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:39:37.715532 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6633528e-b0a1-4f82-858f-c38b791635af admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:37.719341 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8529ab-4560-4ca6-8f6f-73a1133088e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e9ed4d47aef4140bfb2d3bb798cc559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:37.719963 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8529ab-4560-4ca6-8f6f-73a1133088e9 service neutron] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:37.720240 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8529ab-4560-4ca6-8f6f-73a1133088e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:37.720517 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8529ab-4560-4ca6-8f6f-73a1133088e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:37.731882 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8529ab-4560-4ca6-8f6f-73a1133088e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:37.733924 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8529ab-4560-4ca6-8f6f-73a1133088e9 service neutron] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:37.744250 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1316/2655] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:39:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:39:38.211102 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1633ed33-2d06-4f1e-bc4f-2f6bda73ec7d None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:38.212583 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-1633ed33-2d06-4f1e-bc4f-2f6bda73ec7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:39:38.227557 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1633ed33-2d06-4f1e-bc4f-2f6bda73ec7d None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:38.234283 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd0692e-0ff7-4c17-9e4f-a994e0e01450 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:38.235378 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd0692e-0ff7-4c17-9e4f-a994e0e01450 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:38.235615 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd0692e-0ff7-4c17-9e4f-a994e0e01450 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:38.236011 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd0692e-0ff7-4c17-9e4f-a994e0e01450 admin admin] PATH_INFO: `/v3/projects/509c2c7a42784298957c57f2a114babf` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:38.238177 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd0692e-0ff7-4c17-9e4f-a994e0e01450 admin admin] RBAC: Authorizing `identity:delete_project(project_id=509c2c7a42784298957c57f2a114babf)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:38.241638 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd0692e-0ff7-4c17-9e4f-a994e0e01450 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:38.267355 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:39:38.416189 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-ccd0692e-0ff7-4c17-9e4f-a994e0e01450 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '509c2c7a42784298957c57f2a114babf', 'request_id': 'req-ccd0692e-0ff7-4c17-9e4f-a994e0e01450'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:38.537602 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-ccd0692e-0ff7-4c17-9e4f-a994e0e01450 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '509c2c7a42784298957c57f2a114babf', 'request_id': 'req-ccd0692e-0ff7-4c17-9e4f-a994e0e01450'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:38.538754 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-ccd0692e-0ff7-4c17-9e4f-a994e0e01450 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '509c2c7a42784298957c57f2a114babf', 'request_id': 'req-ccd0692e-0ff7-4c17-9e4f-a994e0e01450'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:38.555516 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-ccd0692e-0ff7-4c17-9e4f-a994e0e01450 admin admin] The token cache is being invalidate because project 509c2c7a42784298957c57f2a114babf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81731) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:39:38.555516 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-ccd0692e-0ff7-4c17-9e4f-a994e0e01450 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:38.556672 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-ccd0692e-0ff7-4c17-9e4f-a994e0e01450 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:38.559850 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1340/2656] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 24 17:39:38 2022] DELETE /identity/v3/projects/509c2c7a42784298957c57f2a114babf => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:39:38.564086 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8529ab-4560-4ca6-8f6f-73a1133088e9 service neutron] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:38.574333 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8529ab-4560-4ca6-8f6f-73a1133088e9 service neutron] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:38.613412 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:39:38.643911 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75849293-fcf7-4a93-a295-6ec469f61038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:38.644547 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75849293-fcf7-4a93-a295-6ec469f61038 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:38.644830 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75849293-fcf7-4a93-a295-6ec469f61038 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:38.645105 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75849293-fcf7-4a93-a295-6ec469f61038 admin admin] PATH_INFO: `/v3/users/c8b239a784854858ba1cb01f1f083d8e` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:38.646246 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75849293-fcf7-4a93-a295-6ec469f61038 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8b239a784854858ba1cb01f1f083d8e)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:38.648309 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75849293-fcf7-4a93-a295-6ec469f61038 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:38.863287 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-75849293-fcf7-4a93-a295-6ec469f61038 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8b239a784854858ba1cb01f1f083d8e'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:38.929793 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-75849293-fcf7-4a93-a295-6ec469f61038 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8b239a784854858ba1cb01f1f083d8e'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:38.930436 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-75849293-fcf7-4a93-a295-6ec469f61038 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8b239a784854858ba1cb01f1f083d8e'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:38.939447 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-75849293-fcf7-4a93-a295-6ec469f61038 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8b239a784854858ba1cb01f1f083d8e'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:38.939968 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-75849293-fcf7-4a93-a295-6ec469f61038 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8b239a784854858ba1cb01f1f083d8e'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:38.960128 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1317/2657] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 24 17:39:38 2022] DELETE /identity/v3/users/c8b239a784854858ba1cb01f1f083d8e => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:39:39.571930 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd0692e-0ff7-4c17-9e4f-a994e0e01450 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:39.573466 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-ccd0692e-0ff7-4c17-9e4f-a994e0e01450 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:39:39.590554 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd0692e-0ff7-4c17-9e4f-a994e0e01450 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:39.673899 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f17395-f1d1-4b71-8c7e-1ec47b12afc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:39.675021 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f17395-f1d1-4b71-8c7e-1ec47b12afc9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:39.675520 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f17395-f1d1-4b71-8c7e-1ec47b12afc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:39.676047 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f17395-f1d1-4b71-8c7e-1ec47b12afc9 admin admin] PATH_INFO: `/v3/projects/25999f14ae2245a5a7b8a6075e31d9f5` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:39.678113 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f17395-f1d1-4b71-8c7e-1ec47b12afc9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=25999f14ae2245a5a7b8a6075e31d9f5)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:39.681715 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f17395-f1d1-4b71-8c7e-1ec47b12afc9 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:39.717183 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75849293-fcf7-4a93-a295-6ec469f61038 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:39.717183 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-75849293-fcf7-4a93-a295-6ec469f61038 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:39:39.737563 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75849293-fcf7-4a93-a295-6ec469f61038 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:39.742217 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed2a8e8-cce4-4caa-ac1d-9c24c4b2bdfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:39.742862 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed2a8e8-cce4-4caa-ac1d-9c24c4b2bdfc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:39.743183 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed2a8e8-cce4-4caa-ac1d-9c24c4b2bdfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:39.743460 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed2a8e8-cce4-4caa-ac1d-9c24c4b2bdfc admin admin] PATH_INFO: `/v3/users/804436148e164d46a65373a1f1c9ed58` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:39.744613 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed2a8e8-cce4-4caa-ac1d-9c24c4b2bdfc admin admin] RBAC: Authorizing `identity:delete_user(user_id=804436148e164d46a65373a1f1c9ed58)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:39.746509 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed2a8e8-cce4-4caa-ac1d-9c24c4b2bdfc admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:39.789758 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-54f17395-f1d1-4b71-8c7e-1ec47b12afc9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25999f14ae2245a5a7b8a6075e31d9f5', 'request_id': 'req-54f17395-f1d1-4b71-8c7e-1ec47b12afc9'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:39.843360 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-54f17395-f1d1-4b71-8c7e-1ec47b12afc9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25999f14ae2245a5a7b8a6075e31d9f5', 'request_id': 'req-54f17395-f1d1-4b71-8c7e-1ec47b12afc9'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:39.850819 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-54f17395-f1d1-4b71-8c7e-1ec47b12afc9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25999f14ae2245a5a7b8a6075e31d9f5', 'request_id': 'req-54f17395-f1d1-4b71-8c7e-1ec47b12afc9'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:39.862787 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-54f17395-f1d1-4b71-8c7e-1ec47b12afc9 admin admin] The token cache is being invalidate because project 25999f14ae2245a5a7b8a6075e31d9f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81731) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:39:39.863100 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-54f17395-f1d1-4b71-8c7e-1ec47b12afc9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:39.863756 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-54f17395-f1d1-4b71-8c7e-1ec47b12afc9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:39.865590 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1341/2658] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 24 17:39:39 2022] DELETE /identity/v3/projects/25999f14ae2245a5a7b8a6075e31d9f5 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:39:39.870441 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2200b9-2544-4237-9c1c-df9c998f35a7 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:39.870737 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2200b9-2544-4237-9c1c-df9c998f35a7 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:39.871041 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2200b9-2544-4237-9c1c-df9c998f35a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:39.883325 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-4ed2a8e8-cce4-4caa-ac1d-9c24c4b2bdfc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '804436148e164d46a65373a1f1c9ed58'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:39.891841 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-fe2200b9-2544-4237-9c1c-df9c998f35a7 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:39:39.893080 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-fe2200b9-2544-4237-9c1c-df9c998f35a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:39:39.927759 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:39:39.954238 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1342/2659] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:39:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:39:39.961613 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f10fcb-ffeb-4ab7-be70-884fce000bf9 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:39.961902 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f10fcb-ffeb-4ab7-be70-884fce000bf9 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:39.962297 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f10fcb-ffeb-4ab7-be70-884fce000bf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:39.981321 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-88f10fcb-ffeb-4ab7-be70-884fce000bf9 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:39:39.984822 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-88f10fcb-ffeb-4ab7-be70-884fce000bf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:39:39.991260 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1343/2660] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:39:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:39:39.995483 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f10fcb-ffeb-4ab7-be70-884fce000bf9 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:40.005634 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f10fcb-ffeb-4ab7-be70-884fce000bf9 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:40.009170 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-4ed2a8e8-cce4-4caa-ac1d-9c24c4b2bdfc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '804436148e164d46a65373a1f1c9ed58'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:40.009822 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-4ed2a8e8-cce4-4caa-ac1d-9c24c4b2bdfc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '804436148e164d46a65373a1f1c9ed58'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:40.011329 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577476d2-d4e4-407e-b974-4067b0d6d228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:40.011939 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577476d2-d4e4-407e-b974-4067b0d6d228 admin admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:40.012222 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577476d2-d4e4-407e-b974-4067b0d6d228 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:40.012495 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577476d2-d4e4-407e-b974-4067b0d6d228 admin admin] PATH_INFO: `/v3/domains` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:40.013145 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577476d2-d4e4-407e-b974-4067b0d6d228 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:39:40.013506 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577476d2-d4e4-407e-b974-4067b0d6d228 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:40.015234 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577476d2-d4e4-407e-b974-4067b0d6d228 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:40.019230 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-4ed2a8e8-cce4-4caa-ac1d-9c24c4b2bdfc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '804436148e164d46a65373a1f1c9ed58'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:40.019402 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-4ed2a8e8-cce4-4caa-ac1d-9c24c4b2bdfc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '804436148e164d46a65373a1f1c9ed58'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:40.031319 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1344/2661] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 24 17:39:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:39:40.032544 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577476d2-d4e4-407e-b974-4067b0d6d228 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:40.034293 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-577476d2-d4e4-407e-b974-4067b0d6d228 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:39:40.038316 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1318/2662] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 24 17:39:39 2022] DELETE /identity/v3/users/804436148e164d46a65373a1f1c9ed58 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:39:40.042651 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5edfe7-2dcc-41a0-bd6a-6650e4fcb15a None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:40.047969 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5edfe7-2dcc-41a0-bd6a-6650e4fcb15a None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:40.047969 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5edfe7-2dcc-41a0-bd6a-6650e4fcb15a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:40.048475 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577476d2-d4e4-407e-b974-4067b0d6d228 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:40.063642 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-9c5edfe7-2dcc-41a0-bd6a-6650e4fcb15a None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:39:40.065182 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-9c5edfe7-2dcc-41a0-bd6a-6650e4fcb15a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:39:40.094630 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:39:40.115396 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e405106-90c5-4877-9d49-7b0503af2a61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:40.116018 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e405106-90c5-4877-9d49-7b0503af2a61 admin admin] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:40.116298 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e405106-90c5-4877-9d49-7b0503af2a61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:40.116569 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e405106-90c5-4877-9d49-7b0503af2a61 admin admin] PATH_INFO: `/v3/projects` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:40.117226 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e405106-90c5-4877-9d49-7b0503af2a61 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:40.119276 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e405106-90c5-4877-9d49-7b0503af2a61 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:40.132315 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1319/2663] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:39:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:39:40.144209 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5edfe7-2dcc-41a0-bd6a-6650e4fcb15a None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:40.147539 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-9c5edfe7-2dcc-41a0-bd6a-6650e4fcb15a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:39:40.228817 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1345/2664] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 24 17:39:40 2022] POST /identity/v3/projects => generated 383 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:39:40.233083 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e405106-90c5-4877-9d49-7b0503af2a61 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:40.245868 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5edfe7-2dcc-41a0-bd6a-6650e4fcb15a None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:40.299006 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e405106-90c5-4877-9d49-7b0503af2a61 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:40.300826 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc56a7e-f9e6-4e3c-94ae-c1acda03d66b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e9ed4d47aef4140bfb2d3bb798cc559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:40.301446 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc56a7e-f9e6-4e3c-94ae-c1acda03d66b service neutron] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:40.301728 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc56a7e-f9e6-4e3c-94ae-c1acda03d66b service neutron] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:40.302003 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc56a7e-f9e6-4e3c-94ae-c1acda03d66b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:40.304419 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffe9d9f-70b1-4b72-a356-96b6c125a302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:40.305106 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffe9d9f-70b1-4b72-a356-96b6c125a302 admin admin] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:40.305352 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffe9d9f-70b1-4b72-a356-96b6c125a302 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:40.305630 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffe9d9f-70b1-4b72-a356-96b6c125a302 admin admin] PATH_INFO: `/v3/users` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:40.306279 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffe9d9f-70b1-4b72-a356-96b6c125a302 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:40.308353 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffe9d9f-70b1-4b72-a356-96b6c125a302 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:40.312747 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc56a7e-f9e6-4e3c-94ae-c1acda03d66b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:40.314081 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc56a7e-f9e6-4e3c-94ae-c1acda03d66b service neutron] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:40.324600 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1320/2665] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:39:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:39:40.405053 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1346/2666] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 24 17:39:40 2022] POST /identity/v3/users => generated 292 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:39:40.409453 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc56a7e-f9e6-4e3c-94ae-c1acda03d66b service neutron] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:40.420346 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc56a7e-f9e6-4e3c-94ae-c1acda03d66b service neutron] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:40.425271 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592100b0-27f8-459f-8a17-2a2a8213a0f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:40.425930 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592100b0-27f8-459f-8a17-2a2a8213a0f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:40.426207 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592100b0-27f8-459f-8a17-2a2a8213a0f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:40.426481 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592100b0-27f8-459f-8a17-2a2a8213a0f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:40.427251 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592100b0-27f8-459f-8a17-2a2a8213a0f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:40.428801 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592100b0-27f8-459f-8a17-2a2a8213a0f1 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:40.439188 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1321/2667] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:39:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:39:40.443434 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffe9d9f-70b1-4b72-a356-96b6c125a302 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:40.454038 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffe9d9f-70b1-4b72-a356-96b6c125a302 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:40.458664 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0473c9-170a-46dc-9e34-a2939f95d351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:40.459275 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0473c9-170a-46dc-9e34-a2939f95d351 admin admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:40.459618 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0473c9-170a-46dc-9e34-a2939f95d351 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:40.459877 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0473c9-170a-46dc-9e34-a2939f95d351 admin admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:40.460520 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0473c9-170a-46dc-9e34-a2939f95d351 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:40.462099 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0473c9-170a-46dc-9e34-a2939f95d351 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:40.472912 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1347/2668] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:39:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:39:40.477120 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592100b0-27f8-459f-8a17-2a2a8213a0f1 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:40.487770 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592100b0-27f8-459f-8a17-2a2a8213a0f1 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:40.492933 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c7c09c-e208-417a-a772-28ba37258e64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:40.493590 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c7c09c-e208-417a-a772-28ba37258e64 admin admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:40.493871 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c7c09c-e208-417a-a772-28ba37258e64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:40.494143 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c7c09c-e208-417a-a772-28ba37258e64 admin admin] PATH_INFO: `/v3/projects/46eaa620363e4eb28ce2c66c47467319/users/3599906dcff340cf9e08d398372d7af1/roles/60dc7e16aee746d5a3e8aa27d7d8525c` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:40.509245 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c7c09c-e208-417a-a772-28ba37258e64 admin admin] RBAC: Authorizing `identity:create_grant(project_id=46eaa620363e4eb28ce2c66c47467319, user_id=3599906dcff340cf9e08d398372d7af1, role_id=60dc7e16aee746d5a3e8aa27d7d8525c)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:40.511971 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c7c09c-e208-417a-a772-28ba37258e64 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:40.605341 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1322/2669] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 24 17:39:40 2022] PUT /identity/v3/projects/46eaa620363e4eb28ce2c66c47467319/users/3599906dcff340cf9e08d398372d7af1/roles/60dc7e16aee746d5a3e8aa27d7d8525c => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:39:40.616453 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec72052-0cf9-47dd-8be7-ff55290d480c None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:40.617576 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec72052-0cf9-47dd-8be7-ff55290d480c None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:40.617576 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec72052-0cf9-47dd-8be7-ff55290d480c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:40.651355 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-0ec72052-0cf9-47dd-8be7-ff55290d480c None None] MFA Rules not processed for user `3599906dcff340cf9e08d398372d7af1`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:39:40.651355 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-0ec72052-0cf9-47dd-8be7-ff55290d480c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:39:40.660957 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:39:40.717760 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1348/2670] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:39:40 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:39:40.728462 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c7c09c-e208-417a-a772-28ba37258e64 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:40.730173 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-e8c7c09c-e208-417a-a772-28ba37258e64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:39:40.760114 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec72052-0cf9-47dd-8be7-ff55290d480c None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:40.761816 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-0ec72052-0cf9-47dd-8be7-ff55290d480c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:39:40.802242 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c7c09c-e208-417a-a772-28ba37258e64 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:40.809436 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41e5a83-c149-4aea-8a2d-0b776122c371 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be06246d47584b529150ceb98a5cfd95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:40.809436 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41e5a83-c149-4aea-8a2d-0b776122c371 service cinder] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:40.809436 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41e5a83-c149-4aea-8a2d-0b776122c371 service cinder] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:40.809436 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41e5a83-c149-4aea-8a2d-0b776122c371 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:40.818614 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:39:40.822637 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41e5a83-c149-4aea-8a2d-0b776122c371 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:40.831627 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41e5a83-c149-4aea-8a2d-0b776122c371 service cinder] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:40.849201 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1323/2671] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:39:40 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:39:40.851385 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec72052-0cf9-47dd-8be7-ff55290d480c None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:40.856319 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc29274-acd4-4a4a-863b-ddf487c0731b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:40.856984 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc29274-acd4-4a4a-863b-ddf487c0731b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:40.857306 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc29274-acd4-4a4a-863b-ddf487c0731b admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:40.857709 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc29274-acd4-4a4a-863b-ddf487c0731b admin admin] PATH_INFO: `/v3/projects/727788f6028948f89d3ab8338e15ba09` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:40.858808 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc29274-acd4-4a4a-863b-ddf487c0731b admin admin] RBAC: Authorizing `identity:delete_project(project_id=727788f6028948f89d3ab8338e15ba09)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:40.860806 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc29274-acd4-4a4a-863b-ddf487c0731b admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:41.009875 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-7dc29274-acd4-4a4a-863b-ddf487c0731b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '727788f6028948f89d3ab8338e15ba09', 'request_id': 'req-7dc29274-acd4-4a4a-863b-ddf487c0731b'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:41.106667 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-7dc29274-acd4-4a4a-863b-ddf487c0731b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '727788f6028948f89d3ab8338e15ba09', 'request_id': 'req-7dc29274-acd4-4a4a-863b-ddf487c0731b'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:41.117536 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-7dc29274-acd4-4a4a-863b-ddf487c0731b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '727788f6028948f89d3ab8338e15ba09', 'request_id': 'req-7dc29274-acd4-4a4a-863b-ddf487c0731b'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:41.132790 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-7dc29274-acd4-4a4a-863b-ddf487c0731b admin admin] The token cache is being invalidate because project 727788f6028948f89d3ab8338e15ba09 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81731) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:39:41.133082 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-7dc29274-acd4-4a4a-863b-ddf487c0731b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:41.133773 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-7dc29274-acd4-4a4a-863b-ddf487c0731b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:41.135379 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1349/2672] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 24 17:39:40 2022] DELETE /identity/v3/projects/727788f6028948f89d3ab8338e15ba09 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:39:41.140824 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80de9751-84bb-47ae-8586-7cf8a18bfd11 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:41.156659 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80de9751-84bb-47ae-8586-7cf8a18bfd11 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:41.156659 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80de9751-84bb-47ae-8586-7cf8a18bfd11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:41.162062 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-80de9751-84bb-47ae-8586-7cf8a18bfd11 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:39:41.163342 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-80de9751-84bb-47ae-8586-7cf8a18bfd11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:39:41.232873 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1324/2673] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:39:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:39:41.241229 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6fe0e2-fa8e-45f2-a4f9-a0584311712e None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:41.241607 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6fe0e2-fa8e-45f2-a4f9-a0584311712e None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:41.242068 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6fe0e2-fa8e-45f2-a4f9-a0584311712e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:41.265366 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-cb6fe0e2-fa8e-45f2-a4f9-a0584311712e None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:39:41.268834 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-cb6fe0e2-fa8e-45f2-a4f9-a0584311712e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:39:41.277322 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1350/2674] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:39:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:39:41.282478 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80de9751-84bb-47ae-8586-7cf8a18bfd11 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:41.292377 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80de9751-84bb-47ae-8586-7cf8a18bfd11 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:41.297180 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6593fa74-79ba-493e-9c63-2f950b54edf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:41.297781 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6593fa74-79ba-493e-9c63-2f950b54edf9 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:41.298056 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6593fa74-79ba-493e-9c63-2f950b54edf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:41.298326 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6593fa74-79ba-493e-9c63-2f950b54edf9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:41.299139 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6593fa74-79ba-493e-9c63-2f950b54edf9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:39:41.299444 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6593fa74-79ba-493e-9c63-2f950b54edf9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:41.301030 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6593fa74-79ba-493e-9c63-2f950b54edf9 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:41.313979 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1325/2675] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 24 17:39:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:39:41.318425 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6fe0e2-fa8e-45f2-a4f9-a0584311712e None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:41.328416 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6fe0e2-fa8e-45f2-a4f9-a0584311712e None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:41.336197 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe11a36-f364-4ac4-861a-a6aeeb1805c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:41.336797 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe11a36-f364-4ac4-861a-a6aeeb1805c8 admin admin] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:41.337215 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe11a36-f364-4ac4-861a-a6aeeb1805c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:41.337491 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe11a36-f364-4ac4-861a-a6aeeb1805c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:41.338125 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe11a36-f364-4ac4-861a-a6aeeb1805c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:41.340085 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe11a36-f364-4ac4-861a-a6aeeb1805c8 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:41.446007 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1351/2676] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 24 17:39:41 2022] POST /identity/v3/projects => generated 385 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:39:41.450421 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6593fa74-79ba-493e-9c63-2f950b54edf9 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:41.462792 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6593fa74-79ba-493e-9c63-2f950b54edf9 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:41.541964 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16e1165-6180-41f8-839a-6e76dea9f2c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:41.542709 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16e1165-6180-41f8-839a-6e76dea9f2c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:41.543052 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16e1165-6180-41f8-839a-6e76dea9f2c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:41.543349 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16e1165-6180-41f8-839a-6e76dea9f2c0 admin admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:41.544013 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16e1165-6180-41f8-839a-6e76dea9f2c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:41.546239 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16e1165-6180-41f8-839a-6e76dea9f2c0 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:41.667015 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1326/2677] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 24 17:39:41 2022] POST /identity/v3/users => generated 293 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:39:41.671260 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe11a36-f364-4ac4-861a-a6aeeb1805c8 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:41.702190 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe11a36-f364-4ac4-861a-a6aeeb1805c8 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:41.702190 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa63469a-b38b-4da9-8ef8-8046379e91c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:41.702190 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa63469a-b38b-4da9-8ef8-8046379e91c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:41.702190 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa63469a-b38b-4da9-8ef8-8046379e91c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:41.702615 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa63469a-b38b-4da9-8ef8-8046379e91c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:41.702615 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa63469a-b38b-4da9-8ef8-8046379e91c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:41.704259 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa63469a-b38b-4da9-8ef8-8046379e91c7 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:41.718922 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1352/2678] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:39:41 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:39:41.723099 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16e1165-6180-41f8-839a-6e76dea9f2c0 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:41.735872 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16e1165-6180-41f8-839a-6e76dea9f2c0 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:41.744249 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293ad362-a2e9-448c-839f-f1bf79ac031c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:41.744943 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293ad362-a2e9-448c-839f-f1bf79ac031c admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:41.745292 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293ad362-a2e9-448c-839f-f1bf79ac031c admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:41.745639 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293ad362-a2e9-448c-839f-f1bf79ac031c admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:41.746299 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293ad362-a2e9-448c-839f-f1bf79ac031c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:41.748512 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293ad362-a2e9-448c-839f-f1bf79ac031c admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:41.760619 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1327/2679] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:39:41 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:39:41.766525 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa63469a-b38b-4da9-8ef8-8046379e91c7 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:41.772794 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:39:41.778857 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa63469a-b38b-4da9-8ef8-8046379e91c7 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:41.784355 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccb0d58-d434-443b-84a4-683756ad8e3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:41.785138 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccb0d58-d434-443b-84a4-683756ad8e3a admin admin] REQUEST_METHOD: `PUT` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:41.785597 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccb0d58-d434-443b-84a4-683756ad8e3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:41.785978 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccb0d58-d434-443b-84a4-683756ad8e3a admin admin] PATH_INFO: `/v3/projects/297528b99cc84668bf23ca032eb49715/users/4335a7fc165e4b67b65a2f26ee1fc1f2/roles/60dc7e16aee746d5a3e8aa27d7d8525c` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:41.804462 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccb0d58-d434-443b-84a4-683756ad8e3a admin admin] RBAC: Authorizing `identity:create_grant(project_id=297528b99cc84668bf23ca032eb49715, user_id=4335a7fc165e4b67b65a2f26ee1fc1f2, role_id=60dc7e16aee746d5a3e8aa27d7d8525c)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:41.807267 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccb0d58-d434-443b-84a4-683756ad8e3a admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:41.924898 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1353/2680] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 24 17:39:41 2022] PUT /identity/v3/projects/297528b99cc84668bf23ca032eb49715/users/4335a7fc165e4b67b65a2f26ee1fc1f2/roles/60dc7e16aee746d5a3e8aa27d7d8525c => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:39:41.935666 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96d96f7-19f4-4a55-a35b-9878c5055494 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:41.936016 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96d96f7-19f4-4a55-a35b-9878c5055494 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:41.936354 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96d96f7-19f4-4a55-a35b-9878c5055494 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:41.968429 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-d96d96f7-19f4-4a55-a35b-9878c5055494 None None] MFA Rules not processed for user `4335a7fc165e4b67b65a2f26ee1fc1f2`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:39:41.969745 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-d96d96f7-19f4-4a55-a35b-9878c5055494 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:39:42.025151 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:39:42.047856 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1328/2681] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:39:41 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:39:42.049102 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccb0d58-d434-443b-84a4-683756ad8e3a admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:42.136006 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccb0d58-d434-443b-84a4-683756ad8e3a admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:42.142173 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ced300-fff2-40de-9342-450629cf86f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:42.142802 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ced300-fff2-40de-9342-450629cf86f3 admin admin] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:42.143113 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ced300-fff2-40de-9342-450629cf86f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:42.143389 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ced300-fff2-40de-9342-450629cf86f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:42.144027 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ced300-fff2-40de-9342-450629cf86f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:42.145952 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ced300-fff2-40de-9342-450629cf86f3 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:42.411542 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1354/2682] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 24 17:39:42 2022] POST /identity/v3/projects => generated 387 bytes in 363 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:39:42.415708 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96d96f7-19f4-4a55-a35b-9878c5055494 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:42.427082 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96d96f7-19f4-4a55-a35b-9878c5055494 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:42.487027 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3049126-4ffc-4f3f-b000-8985d1f5381b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:42.487644 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3049126-4ffc-4f3f-b000-8985d1f5381b admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:42.487930 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3049126-4ffc-4f3f-b000-8985d1f5381b admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:42.488199 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3049126-4ffc-4f3f-b000-8985d1f5381b admin admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:42.488869 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3049126-4ffc-4f3f-b000-8985d1f5381b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:42.490849 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3049126-4ffc-4f3f-b000-8985d1f5381b admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:42.621413 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1329/2683] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 24 17:39:42 2022] POST /identity/v3/users => generated 294 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:39:42.626137 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ced300-fff2-40de-9342-450629cf86f3 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:42.637014 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ced300-fff2-40de-9342-450629cf86f3 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:42.649353 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a877780-d66f-434c-a7d7-eb8462f1084e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:42.649353 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a877780-d66f-434c-a7d7-eb8462f1084e admin admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:42.649353 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a877780-d66f-434c-a7d7-eb8462f1084e admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:42.649353 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a877780-d66f-434c-a7d7-eb8462f1084e admin admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:42.649850 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a877780-d66f-434c-a7d7-eb8462f1084e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:42.649850 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a877780-d66f-434c-a7d7-eb8462f1084e admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:42.657330 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1355/2684] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:39:42 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:39:42.661600 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3049126-4ffc-4f3f-b000-8985d1f5381b admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:42.672407 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3049126-4ffc-4f3f-b000-8985d1f5381b admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:42.677083 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7395e93-5b0b-49d9-ad28-c7681ba1b2cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:42.677698 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7395e93-5b0b-49d9-ad28-c7681ba1b2cc admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:42.677977 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7395e93-5b0b-49d9-ad28-c7681ba1b2cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:42.678250 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7395e93-5b0b-49d9-ad28-c7681ba1b2cc admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:42.678891 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7395e93-5b0b-49d9-ad28-c7681ba1b2cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:42.685237 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7395e93-5b0b-49d9-ad28-c7681ba1b2cc admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:42.695457 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1330/2685] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:39:42 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:39:42.699627 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a877780-d66f-434c-a7d7-eb8462f1084e admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:42.712448 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a877780-d66f-434c-a7d7-eb8462f1084e admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:42.719768 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d740bcf9-c225-4ab8-b99b-78ae619e1d67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:42.729599 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d740bcf9-c225-4ab8-b99b-78ae619e1d67 admin admin] REQUEST_METHOD: `PUT` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:42.729599 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d740bcf9-c225-4ab8-b99b-78ae619e1d67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:42.729599 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d740bcf9-c225-4ab8-b99b-78ae619e1d67 admin admin] PATH_INFO: `/v3/projects/5842527515514eebaac1bffccc3561fc/users/9f9583f4a22f463695bee65c683b8163/roles/60dc7e16aee746d5a3e8aa27d7d8525c` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:42.739760 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d740bcf9-c225-4ab8-b99b-78ae619e1d67 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5842527515514eebaac1bffccc3561fc, user_id=9f9583f4a22f463695bee65c683b8163, role_id=60dc7e16aee746d5a3e8aa27d7d8525c)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:42.742477 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d740bcf9-c225-4ab8-b99b-78ae619e1d67 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:42.863851 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1356/2686] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 24 17:39:42 2022] PUT /identity/v3/projects/5842527515514eebaac1bffccc3561fc/users/9f9583f4a22f463695bee65c683b8163/roles/60dc7e16aee746d5a3e8aa27d7d8525c => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:39:42.878728 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049afb64-0447-44e9-8668-d34bd35c7fbd None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:42.879430 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049afb64-0447-44e9-8668-d34bd35c7fbd None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:42.879430 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049afb64-0447-44e9-8668-d34bd35c7fbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:42.910121 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-049afb64-0447-44e9-8668-d34bd35c7fbd None None] MFA Rules not processed for user `9f9583f4a22f463695bee65c683b8163`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:39:42.911370 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-049afb64-0447-44e9-8668-d34bd35c7fbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:39:42.989113 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1331/2687] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:39:42 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:39:42.993945 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d740bcf9-c225-4ab8-b99b-78ae619e1d67 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:43.065358 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d740bcf9-c225-4ab8-b99b-78ae619e1d67 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:43.070158 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e422f0-e5e6-454a-bc95-fa2f5550b2f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:43.070774 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e422f0-e5e6-454a-bc95-fa2f5550b2f5 admin admin] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:43.071102 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e422f0-e5e6-454a-bc95-fa2f5550b2f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:43.071378 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e422f0-e5e6-454a-bc95-fa2f5550b2f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:43.072012 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e422f0-e5e6-454a-bc95-fa2f5550b2f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:43.073947 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e422f0-e5e6-454a-bc95-fa2f5550b2f5 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:43.165720 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1357/2688] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 24 17:39:42 2022] POST /identity/v3/projects => generated 385 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:39:43.170111 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049afb64-0447-44e9-8668-d34bd35c7fbd None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:43.181951 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049afb64-0447-44e9-8668-d34bd35c7fbd None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:43.249634 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e64ab1e-0499-41fb-aa19-25b88720aff5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:43.250250 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e64ab1e-0499-41fb-aa19-25b88720aff5 admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:43.250532 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e64ab1e-0499-41fb-aa19-25b88720aff5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:43.250802 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e64ab1e-0499-41fb-aa19-25b88720aff5 admin admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:43.251523 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e64ab1e-0499-41fb-aa19-25b88720aff5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:43.253369 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e64ab1e-0499-41fb-aa19-25b88720aff5 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:43.261282 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:39:43.359173 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1332/2689] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 24 17:39:43 2022] POST /identity/v3/users => generated 299 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:39:43.363012 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e422f0-e5e6-454a-bc95-fa2f5550b2f5 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:43.374035 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e422f0-e5e6-454a-bc95-fa2f5550b2f5 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:43.378632 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff77f22f-09c2-4583-a261-c63d11436f18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:43.379345 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff77f22f-09c2-4583-a261-c63d11436f18 admin admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:43.379557 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff77f22f-09c2-4583-a261-c63d11436f18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:43.379829 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff77f22f-09c2-4583-a261-c63d11436f18 admin admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:43.380518 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff77f22f-09c2-4583-a261-c63d11436f18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:43.382247 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff77f22f-09c2-4583-a261-c63d11436f18 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:43.393901 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1358/2690] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:39:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:39:43.397913 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e64ab1e-0499-41fb-aa19-25b88720aff5 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:43.412362 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e64ab1e-0499-41fb-aa19-25b88720aff5 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:43.417047 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808f3861-39d5-4e0b-87e7-9046218c38c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:43.417767 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808f3861-39d5-4e0b-87e7-9046218c38c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:43.418027 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808f3861-39d5-4e0b-87e7-9046218c38c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:43.418297 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808f3861-39d5-4e0b-87e7-9046218c38c2 admin admin] PATH_INFO: `/v3/projects/4e66eb8c329647619eaeb94dc1de342e/users/0bc30c70ea7941ada78ebc5db0045fa2/roles/c5346bc54abf445f9900edf94264c850` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:43.433121 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808f3861-39d5-4e0b-87e7-9046218c38c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e66eb8c329647619eaeb94dc1de342e, user_id=0bc30c70ea7941ada78ebc5db0045fa2, role_id=c5346bc54abf445f9900edf94264c850)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:43.436332 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808f3861-39d5-4e0b-87e7-9046218c38c2 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:43.541138 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1333/2691] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 24 17:39:43 2022] PUT /identity/v3/projects/4e66eb8c329647619eaeb94dc1de342e/users/0bc30c70ea7941ada78ebc5db0045fa2/roles/c5346bc54abf445f9900edf94264c850 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:39:43.546531 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff77f22f-09c2-4583-a261-c63d11436f18 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:43.556049 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:39:43.557445 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff77f22f-09c2-4583-a261-c63d11436f18 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:43.621172 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731a595d-925d-4448-992f-a6f54651d5a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:43.621761 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731a595d-925d-4448-992f-a6f54651d5a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:43.622042 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731a595d-925d-4448-992f-a6f54651d5a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:43.622311 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731a595d-925d-4448-992f-a6f54651d5a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:43.623108 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731a595d-925d-4448-992f-a6f54651d5a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:43.624709 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731a595d-925d-4448-992f-a6f54651d5a8 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:43.636636 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1359/2692] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:39:43 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:39:43.641546 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808f3861-39d5-4e0b-87e7-9046218c38c2 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:43.652999 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808f3861-39d5-4e0b-87e7-9046218c38c2 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:43.659918 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0130c7-af97-4e01-a4a2-7dd179f55ca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:43.659918 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0130c7-af97-4e01-a4a2-7dd179f55ca4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:43.660410 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0130c7-af97-4e01-a4a2-7dd179f55ca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:43.660720 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0130c7-af97-4e01-a4a2-7dd179f55ca4 admin admin] PATH_INFO: `/v3/projects/4e66eb8c329647619eaeb94dc1de342e/users/0bc30c70ea7941ada78ebc5db0045fa2/roles/c5346bc54abf445f9900edf94264c850` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:43.662795 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0130c7-af97-4e01-a4a2-7dd179f55ca4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e66eb8c329647619eaeb94dc1de342e, user_id=0bc30c70ea7941ada78ebc5db0045fa2, role_id=c5346bc54abf445f9900edf94264c850)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:43.665773 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0130c7-af97-4e01-a4a2-7dd179f55ca4 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:43.680271 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1334/2693] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 24 17:39:43 2022] PUT /identity/v3/projects/4e66eb8c329647619eaeb94dc1de342e/users/0bc30c70ea7941ada78ebc5db0045fa2/roles/c5346bc54abf445f9900edf94264c850 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:39:43.689822 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301378ca-3d37-447a-90ea-0c88e4b28e1e None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:43.690112 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301378ca-3d37-447a-90ea-0c88e4b28e1e None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:43.690836 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301378ca-3d37-447a-90ea-0c88e4b28e1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:43.721226 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-301378ca-3d37-447a-90ea-0c88e4b28e1e None None] MFA Rules not processed for user `0bc30c70ea7941ada78ebc5db0045fa2`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:39:43.722448 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-301378ca-3d37-447a-90ea-0c88e4b28e1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:39:43.792962 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1360/2694] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:39:43 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:39:43.803859 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0130c7-af97-4e01-a4a2-7dd179f55ca4 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:43.805592 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-3a0130c7-af97-4e01-a4a2-7dd179f55ca4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:39:43.876349 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0130c7-af97-4e01-a4a2-7dd179f55ca4 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:43.880114 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4623b6-14ee-4560-b40c-2250828b7ecd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be06246d47584b529150ceb98a5cfd95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:43.880740 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4623b6-14ee-4560-b40c-2250828b7ecd service cinder] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:43.880987 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4623b6-14ee-4560-b40c-2250828b7ecd service cinder] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:43.881256 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4623b6-14ee-4560-b40c-2250828b7ecd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:43.891633 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4623b6-14ee-4560-b40c-2250828b7ecd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:43.893118 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4623b6-14ee-4560-b40c-2250828b7ecd service cinder] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:43.904352 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1335/2695] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:39:43 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:39:45.086169 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301378ca-3d37-447a-90ea-0c88e4b28e1e None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:45.097534 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301378ca-3d37-447a-90ea-0c88e4b28e1e None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:45.101514 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b06a50b-880e-4a75-a9a6-c2e7f08c6cb9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be06246d47584b529150ceb98a5cfd95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:45.105177 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b06a50b-880e-4a75-a9a6-c2e7f08c6cb9 service cinder] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:45.105177 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b06a50b-880e-4a75-a9a6-c2e7f08c6cb9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:45.105177 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b06a50b-880e-4a75-a9a6-c2e7f08c6cb9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:45.166292 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b06a50b-880e-4a75-a9a6-c2e7f08c6cb9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:45.167841 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b06a50b-880e-4a75-a9a6-c2e7f08c6cb9 service cinder] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:45.178357 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1361/2696] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:39:45 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:39:53.349493 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4623b6-14ee-4560-b40c-2250828b7ecd service cinder] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:53.351214 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-5d4623b6-14ee-4560-b40c-2250828b7ecd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:39:53.379720 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b06a50b-880e-4a75-a9a6-c2e7f08c6cb9 service cinder] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:53.389139 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b06a50b-880e-4a75-a9a6-c2e7f08c6cb9 service cinder] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:53.394976 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f2726a-10ff-4e18-a3e8-449fe42da104 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be06246d47584b529150ceb98a5cfd95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:53.397201 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f2726a-10ff-4e18-a3e8-449fe42da104 service cinder] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:53.397201 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f2726a-10ff-4e18-a3e8-449fe42da104 service cinder] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:53.397201 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f2726a-10ff-4e18-a3e8-449fe42da104 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:53.416938 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:39:53.423869 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4623b6-14ee-4560-b40c-2250828b7ecd service cinder] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:53.428864 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0914ba7f-6676-4b6f-b7df-fcd0ceceef22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:53.431579 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0914ba7f-6676-4b6f-b7df-fcd0ceceef22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:53.431579 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0914ba7f-6676-4b6f-b7df-fcd0ceceef22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:53.431579 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0914ba7f-6676-4b6f-b7df-fcd0ceceef22 admin admin] PATH_INFO: `/v3/users/3599906dcff340cf9e08d398372d7af1` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:53.431579 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0914ba7f-6676-4b6f-b7df-fcd0ceceef22 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3599906dcff340cf9e08d398372d7af1)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:53.433330 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0914ba7f-6676-4b6f-b7df-fcd0ceceef22 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:53.461552 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f2726a-10ff-4e18-a3e8-449fe42da104 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:53.463060 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f2726a-10ff-4e18-a3e8-449fe42da104 service cinder] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:53.472098 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1362/2697] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:39:53 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:39:53.698650 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-0914ba7f-6676-4b6f-b7df-fcd0ceceef22 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3599906dcff340cf9e08d398372d7af1'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:53.781757 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-0914ba7f-6676-4b6f-b7df-fcd0ceceef22 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3599906dcff340cf9e08d398372d7af1'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:53.782547 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-0914ba7f-6676-4b6f-b7df-fcd0ceceef22 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3599906dcff340cf9e08d398372d7af1'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:53.785391 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:INSERT=6,keystone:SELECT=86,keystone:DELETE=8 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:39:53.792691 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-0914ba7f-6676-4b6f-b7df-fcd0ceceef22 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3599906dcff340cf9e08d398372d7af1'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:53.793588 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-0914ba7f-6676-4b6f-b7df-fcd0ceceef22 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3599906dcff340cf9e08d398372d7af1'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:53.814186 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1336/2698] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 24 17:39:53 2022] DELETE /identity/v3/users/3599906dcff340cf9e08d398372d7af1 => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:39:53.818537 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37546524-6fae-48ef-9c2d-21299d69270f None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:53.818824 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37546524-6fae-48ef-9c2d-21299d69270f None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:53.819132 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37546524-6fae-48ef-9c2d-21299d69270f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:53.839881 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-37546524-6fae-48ef-9c2d-21299d69270f None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:39:53.840320 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-37546524-6fae-48ef-9c2d-21299d69270f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:39:53.907504 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1363/2699] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:39:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:39:53.915841 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0914ba7f-6676-4b6f-b7df-fcd0ceceef22 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:53.919272 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-0914ba7f-6676-4b6f-b7df-fcd0ceceef22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:39:53.990857 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0914ba7f-6676-4b6f-b7df-fcd0ceceef22 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:53.995192 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfccb50-be13-4b4b-9142-4627aff361a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e9ed4d47aef4140bfb2d3bb798cc559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:53.995833 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfccb50-be13-4b4b-9142-4627aff361a7 service neutron] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:53.996118 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfccb50-be13-4b4b-9142-4627aff361a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:53.996389 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfccb50-be13-4b4b-9142-4627aff361a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:54.007951 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfccb50-be13-4b4b-9142-4627aff361a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:54.012826 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfccb50-be13-4b4b-9142-4627aff361a7 service neutron] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:54.025081 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1337/2700] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:39:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:39:54.503155 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37546524-6fae-48ef-9c2d-21299d69270f None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:54.506726 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-37546524-6fae-48ef-9c2d-21299d69270f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:39:54.518062 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37546524-6fae-48ef-9c2d-21299d69270f None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:54.526026 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d30ed04-b85b-477e-90ec-3fbb75a809e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:54.526710 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d30ed04-b85b-477e-90ec-3fbb75a809e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:54.527112 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d30ed04-b85b-477e-90ec-3fbb75a809e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:54.527391 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d30ed04-b85b-477e-90ec-3fbb75a809e4 admin admin] PATH_INFO: `/v3/projects/46eaa620363e4eb28ce2c66c47467319` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:54.528860 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d30ed04-b85b-477e-90ec-3fbb75a809e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=46eaa620363e4eb28ce2c66c47467319)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:54.531083 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d30ed04-b85b-477e-90ec-3fbb75a809e4 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:54.677120 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-9d30ed04-b85b-477e-90ec-3fbb75a809e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46eaa620363e4eb28ce2c66c47467319', 'request_id': 'req-9d30ed04-b85b-477e-90ec-3fbb75a809e4'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:54.758491 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-9d30ed04-b85b-477e-90ec-3fbb75a809e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46eaa620363e4eb28ce2c66c47467319', 'request_id': 'req-9d30ed04-b85b-477e-90ec-3fbb75a809e4'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:54.768938 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-9d30ed04-b85b-477e-90ec-3fbb75a809e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46eaa620363e4eb28ce2c66c47467319', 'request_id': 'req-9d30ed04-b85b-477e-90ec-3fbb75a809e4'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:54.780721 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-9d30ed04-b85b-477e-90ec-3fbb75a809e4 admin admin] The token cache is being invalidate because project 46eaa620363e4eb28ce2c66c47467319 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81731) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:39:54.780924 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-9d30ed04-b85b-477e-90ec-3fbb75a809e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:54.782079 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-9d30ed04-b85b-477e-90ec-3fbb75a809e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:54.783633 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1364/2701] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 24 17:39:54 2022] DELETE /identity/v3/projects/46eaa620363e4eb28ce2c66c47467319 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:39:54.788870 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2ce177-92d6-448d-b859-2f3873517fd3 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:54.789372 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2ce177-92d6-448d-b859-2f3873517fd3 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:54.789474 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2ce177-92d6-448d-b859-2f3873517fd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:54.810977 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-7d2ce177-92d6-448d-b859-2f3873517fd3 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:39:54.812237 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-7d2ce177-92d6-448d-b859-2f3873517fd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:39:54.874138 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1338/2702] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:39:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:39:54.883261 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5fcc2f-bfa9-4744-a773-3329299ef09d None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:54.889361 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5fcc2f-bfa9-4744-a773-3329299ef09d None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:54.889361 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5fcc2f-bfa9-4744-a773-3329299ef09d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:54.908558 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-6f5fcc2f-bfa9-4744-a773-3329299ef09d None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:39:54.909771 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-6f5fcc2f-bfa9-4744-a773-3329299ef09d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:39:54.920471 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1365/2703] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:39:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:39:54.925580 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2ce177-92d6-448d-b859-2f3873517fd3 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:54.936250 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2ce177-92d6-448d-b859-2f3873517fd3 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:54.941009 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5f0f2d-a7b3-42f4-98a1-9a80954be61e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:54.941622 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5f0f2d-a7b3-42f4-98a1-9a80954be61e admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:54.941906 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5f0f2d-a7b3-42f4-98a1-9a80954be61e admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:54.942323 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5f0f2d-a7b3-42f4-98a1-9a80954be61e admin admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:54.943012 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5f0f2d-a7b3-42f4-98a1-9a80954be61e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:39:54.943315 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5f0f2d-a7b3-42f4-98a1-9a80954be61e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:54.944967 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5f0f2d-a7b3-42f4-98a1-9a80954be61e admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:54.958370 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1339/2704] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 24 17:39:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:39:54.963018 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5fcc2f-bfa9-4744-a773-3329299ef09d None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:54.975225 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5fcc2f-bfa9-4744-a773-3329299ef09d None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:54.980679 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd19ffab-69b9-46e3-8204-d491a7d5dd8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:54.981306 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd19ffab-69b9-46e3-8204-d491a7d5dd8c admin admin] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:54.981594 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd19ffab-69b9-46e3-8204-d491a7d5dd8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:54.981868 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd19ffab-69b9-46e3-8204-d491a7d5dd8c admin admin] PATH_INFO: `/v3/projects` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:54.982511 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd19ffab-69b9-46e3-8204-d491a7d5dd8c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:54.985086 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd19ffab-69b9-46e3-8204-d491a7d5dd8c admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:55.075067 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1366/2705] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 24 17:39:54 2022] POST /identity/v3/projects => generated 383 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:39:55.081916 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5f0f2d-a7b3-42f4-98a1-9a80954be61e admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:55.095508 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5f0f2d-a7b3-42f4-98a1-9a80954be61e admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:55.161698 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e3d913-10eb-48b2-94eb-bff3e57a4f21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:55.162458 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e3d913-10eb-48b2-94eb-bff3e57a4f21 admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:55.162606 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e3d913-10eb-48b2-94eb-bff3e57a4f21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:55.162873 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e3d913-10eb-48b2-94eb-bff3e57a4f21 admin admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:55.163548 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e3d913-10eb-48b2-94eb-bff3e57a4f21 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:55.165485 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e3d913-10eb-48b2-94eb-bff3e57a4f21 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:55.251591 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:39:55.364596 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1340/2706] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 24 17:39:55 2022] POST /identity/v3/users => generated 292 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:39:55.368963 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd19ffab-69b9-46e3-8204-d491a7d5dd8c admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:55.380244 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd19ffab-69b9-46e3-8204-d491a7d5dd8c admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:55.385029 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a26ec5-fd1c-412f-ac95-1e81d1c99db4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:55.385791 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a26ec5-fd1c-412f-ac95-1e81d1c99db4 admin admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:55.386036 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a26ec5-fd1c-412f-ac95-1e81d1c99db4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:55.386307 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a26ec5-fd1c-412f-ac95-1e81d1c99db4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:55.386984 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a26ec5-fd1c-412f-ac95-1e81d1c99db4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:55.388576 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a26ec5-fd1c-412f-ac95-1e81d1c99db4 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:55.399289 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1367/2707] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:39:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:39:55.409655 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e3d913-10eb-48b2-94eb-bff3e57a4f21 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:55.420069 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e3d913-10eb-48b2-94eb-bff3e57a4f21 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:55.425323 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f357c0f2-7054-4d92-8ca1-3a0089c1f7a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:55.425939 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f357c0f2-7054-4d92-8ca1-3a0089c1f7a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:55.426218 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f357c0f2-7054-4d92-8ca1-3a0089c1f7a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:55.426523 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f357c0f2-7054-4d92-8ca1-3a0089c1f7a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:55.427213 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f357c0f2-7054-4d92-8ca1-3a0089c1f7a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:55.428820 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f357c0f2-7054-4d92-8ca1-3a0089c1f7a4 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:55.440532 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1341/2708] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:39:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:39:55.444869 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a26ec5-fd1c-412f-ac95-1e81d1c99db4 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:55.451634 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:39:55.455725 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a26ec5-fd1c-412f-ac95-1e81d1c99db4 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:55.461221 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5390f0f3-de94-4cd8-8726-69ae941f4298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:55.461893 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5390f0f3-de94-4cd8-8726-69ae941f4298 admin admin] REQUEST_METHOD: `PUT` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:55.462177 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5390f0f3-de94-4cd8-8726-69ae941f4298 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:55.462456 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5390f0f3-de94-4cd8-8726-69ae941f4298 admin admin] PATH_INFO: `/v3/projects/c4d9fbf10f0e451d920dea6dd098224e/users/a09147b8a98049f482b04933f45b69c7/roles/60dc7e16aee746d5a3e8aa27d7d8525c` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:55.480134 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5390f0f3-de94-4cd8-8726-69ae941f4298 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4d9fbf10f0e451d920dea6dd098224e, user_id=a09147b8a98049f482b04933f45b69c7, role_id=60dc7e16aee746d5a3e8aa27d7d8525c)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:55.482993 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5390f0f3-de94-4cd8-8726-69ae941f4298 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:55.581715 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1368/2709] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 24 17:39:55 2022] PUT /identity/v3/projects/c4d9fbf10f0e451d920dea6dd098224e/users/a09147b8a98049f482b04933f45b69c7/roles/60dc7e16aee746d5a3e8aa27d7d8525c => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:39:55.592420 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384e6474-7bf8-4fbf-b38c-df6862efae64 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:55.592707 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384e6474-7bf8-4fbf-b38c-df6862efae64 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:55.592981 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384e6474-7bf8-4fbf-b38c-df6862efae64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:55.629542 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-384e6474-7bf8-4fbf-b38c-df6862efae64 None None] MFA Rules not processed for user `a09147b8a98049f482b04933f45b69c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:39:55.630776 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-384e6474-7bf8-4fbf-b38c-df6862efae64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:39:55.705563 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1342/2710] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:39:55 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:39:55.717029 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5390f0f3-de94-4cd8-8726-69ae941f4298 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:55.719012 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-5390f0f3-de94-4cd8-8726-69ae941f4298 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:39:55.788651 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5390f0f3-de94-4cd8-8726-69ae941f4298 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:55.792412 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4502e6-411d-4e03-8530-0da6afedd7bb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be06246d47584b529150ceb98a5cfd95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:55.793027 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4502e6-411d-4e03-8530-0da6afedd7bb service cinder] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:55.793310 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4502e6-411d-4e03-8530-0da6afedd7bb service cinder] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:55.793588 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4502e6-411d-4e03-8530-0da6afedd7bb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:55.804864 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4502e6-411d-4e03-8530-0da6afedd7bb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:55.808631 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4502e6-411d-4e03-8530-0da6afedd7bb service cinder] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:55.815751 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1369/2711] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:39:55 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:39:58.043677 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384e6474-7bf8-4fbf-b38c-df6862efae64 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:58.045532 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-384e6474-7bf8-4fbf-b38c-df6862efae64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:39:58.051385 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4502e6-411d-4e03-8530-0da6afedd7bb service cinder] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:58.053382 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-bc4502e6-411d-4e03-8530-0da6afedd7bb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:39:58.137072 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4502e6-411d-4e03-8530-0da6afedd7bb service cinder] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:58.143987 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe22702-fbbe-46fe-9540-31d5ac06ffb6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03fc77c6107e430f90f7b6f7cfd4ec02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:58.144611 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe22702-fbbe-46fe-9540-31d5ac06ffb6 service placement] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:58.144904 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe22702-fbbe-46fe-9540-31d5ac06ffb6 service placement] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:58.145464 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe22702-fbbe-46fe-9540-31d5ac06ffb6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:58.148428 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384e6474-7bf8-4fbf-b38c-df6862efae64 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:58.149515 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-2fe22702-fbbe-46fe-9540-31d5ac06ffb6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:39:58.153977 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0b7ef2-6515-4e65-b375-8155ff409895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:58.154806 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0b7ef2-6515-4e65-b375-8155ff409895 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:58.155379 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0b7ef2-6515-4e65-b375-8155ff409895 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:58.155788 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0b7ef2-6515-4e65-b375-8155ff409895 admin admin] PATH_INFO: `/v3/users/4335a7fc165e4b67b65a2f26ee1fc1f2` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:58.157705 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0b7ef2-6515-4e65-b375-8155ff409895 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4335a7fc165e4b67b65a2f26ee1fc1f2)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:58.159787 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0b7ef2-6515-4e65-b375-8155ff409895 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:58.164046 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe22702-fbbe-46fe-9540-31d5ac06ffb6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:58.166423 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe22702-fbbe-46fe-9540-31d5ac06ffb6 service placement] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:58.178252 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1370/2712] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:39:58 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:39:58.368810 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-2e0b7ef2-6515-4e65-b375-8155ff409895 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4335a7fc165e4b67b65a2f26ee1fc1f2'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:58.472361 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-2e0b7ef2-6515-4e65-b375-8155ff409895 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4335a7fc165e4b67b65a2f26ee1fc1f2'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:58.473073 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-2e0b7ef2-6515-4e65-b375-8155ff409895 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4335a7fc165e4b67b65a2f26ee1fc1f2'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:58.484118 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-2e0b7ef2-6515-4e65-b375-8155ff409895 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4335a7fc165e4b67b65a2f26ee1fc1f2'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:58.484278 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-2e0b7ef2-6515-4e65-b375-8155ff409895 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4335a7fc165e4b67b65a2f26ee1fc1f2'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:58.494458 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:INSERT=2,keystone:SELECT=89,keystone:DELETE=9 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:39:58.507514 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1343/2713] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 24 17:39:58 2022] DELETE /identity/v3/users/4335a7fc165e4b67b65a2f26ee1fc1f2 => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:39:58.512233 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5295d7-f3ee-44ba-9161-b23aa8e91cb0 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:58.512547 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5295d7-f3ee-44ba-9161-b23aa8e91cb0 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:58.512825 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5295d7-f3ee-44ba-9161-b23aa8e91cb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:58.537360 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-5e5295d7-f3ee-44ba-9161-b23aa8e91cb0 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:39:58.538714 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-5e5295d7-f3ee-44ba-9161-b23aa8e91cb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:39:58.611774 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1371/2714] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:39:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:39:58.622198 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0b7ef2-6515-4e65-b375-8155ff409895 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:58.623869 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-2e0b7ef2-6515-4e65-b375-8155ff409895 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:39:58.694391 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0b7ef2-6515-4e65-b375-8155ff409895 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:58.698237 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451871e8-9fee-4946-902c-bf7d1c354333 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e9ed4d47aef4140bfb2d3bb798cc559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:58.699111 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451871e8-9fee-4946-902c-bf7d1c354333 service neutron] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:58.699539 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451871e8-9fee-4946-902c-bf7d1c354333 service neutron] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:58.699820 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451871e8-9fee-4946-902c-bf7d1c354333 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:58.710552 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451871e8-9fee-4946-902c-bf7d1c354333 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:58.712110 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451871e8-9fee-4946-902c-bf7d1c354333 service neutron] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:58.724099 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1344/2715] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:39:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:39:59.326510 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5295d7-f3ee-44ba-9161-b23aa8e91cb0 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:59.326510 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-5e5295d7-f3ee-44ba-9161-b23aa8e91cb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:39:59.337189 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5295d7-f3ee-44ba-9161-b23aa8e91cb0 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:59.342132 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125be502-8f5c-482a-b586-472079f1efdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:59.343059 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125be502-8f5c-482a-b586-472079f1efdf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:59.343059 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125be502-8f5c-482a-b586-472079f1efdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:59.343605 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125be502-8f5c-482a-b586-472079f1efdf admin admin] PATH_INFO: `/v3/projects/297528b99cc84668bf23ca032eb49715` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:59.344499 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125be502-8f5c-482a-b586-472079f1efdf admin admin] RBAC: Authorizing `identity:delete_project(project_id=297528b99cc84668bf23ca032eb49715)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:59.346431 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125be502-8f5c-482a-b586-472079f1efdf admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:59.358686 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:39:59.441032 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451871e8-9fee-4946-902c-bf7d1c354333 service neutron] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:59.442200 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-451871e8-9fee-4946-902c-bf7d1c354333 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:39:59.457011 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451871e8-9fee-4946-902c-bf7d1c354333 service neutron] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:59.462456 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c466a71-8fed-4411-8a5a-705d0ef79481 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:59.463225 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c466a71-8fed-4411-8a5a-705d0ef79481 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:59.463513 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c466a71-8fed-4411-8a5a-705d0ef79481 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:59.463999 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c466a71-8fed-4411-8a5a-705d0ef79481 admin admin] PATH_INFO: `/v3/users/a09147b8a98049f482b04933f45b69c7` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:59.465204 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c466a71-8fed-4411-8a5a-705d0ef79481 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a09147b8a98049f482b04933f45b69c7)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:59.467194 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c466a71-8fed-4411-8a5a-705d0ef79481 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:59.513584 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-125be502-8f5c-482a-b586-472079f1efdf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '297528b99cc84668bf23ca032eb49715', 'request_id': 'req-125be502-8f5c-482a-b586-472079f1efdf'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:59.658962 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-125be502-8f5c-482a-b586-472079f1efdf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '297528b99cc84668bf23ca032eb49715', 'request_id': 'req-125be502-8f5c-482a-b586-472079f1efdf'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:59.667566 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-125be502-8f5c-482a-b586-472079f1efdf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '297528b99cc84668bf23ca032eb49715', 'request_id': 'req-125be502-8f5c-482a-b586-472079f1efdf'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:59.681086 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-125be502-8f5c-482a-b586-472079f1efdf admin admin] The token cache is being invalidate because project 297528b99cc84668bf23ca032eb49715 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81731) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:39:59.681350 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-125be502-8f5c-482a-b586-472079f1efdf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:59.681890 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-125be502-8f5c-482a-b586-472079f1efdf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:59.683512 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1372/2716] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 24 17:39:59 2022] DELETE /identity/v3/projects/297528b99cc84668bf23ca032eb49715 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:39:59.686353 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547c9d7d-dbd7-4293-b5b5-f9f4614dfc6b None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:59.686353 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547c9d7d-dbd7-4293-b5b5-f9f4614dfc6b None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:59.686353 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547c9d7d-dbd7-4293-b5b5-f9f4614dfc6b None None] PATH_INFO: `/` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:59.688558 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1373/2717] 10.222.0.18 () {54 vars in 921 bytes} [Wed Aug 24 17:39:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:39:59.689390 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547c9d7d-dbd7-4293-b5b5-f9f4614dfc6b None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:39:59.691180 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-547c9d7d-dbd7-4293-b5b5-f9f4614dfc6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:39:59.769781 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-1c466a71-8fed-4411-8a5a-705d0ef79481 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a09147b8a98049f482b04933f45b69c7'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:59.778286 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547c9d7d-dbd7-4293-b5b5-f9f4614dfc6b None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:39:59.783909 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b077ad9b-cc07-4ba0-8a04-2cf4f63082ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:39:59.784559 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b077ad9b-cc07-4ba0-8a04-2cf4f63082ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:59.784838 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b077ad9b-cc07-4ba0-8a04-2cf4f63082ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:59.785110 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b077ad9b-cc07-4ba0-8a04-2cf4f63082ba admin admin] PATH_INFO: `/v3/users/9f9583f4a22f463695bee65c683b8163` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:59.786276 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b077ad9b-cc07-4ba0-8a04-2cf4f63082ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f9583f4a22f463695bee65c683b8163)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:39:59.788259 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b077ad9b-cc07-4ba0-8a04-2cf4f63082ba admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:39:59.901077 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-1c466a71-8fed-4411-8a5a-705d0ef79481 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a09147b8a98049f482b04933f45b69c7'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:59.901701 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-1c466a71-8fed-4411-8a5a-705d0ef79481 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a09147b8a98049f482b04933f45b69c7'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:59.909655 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-1c466a71-8fed-4411-8a5a-705d0ef79481 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a09147b8a98049f482b04933f45b69c7'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:59.910369 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-1c466a71-8fed-4411-8a5a-705d0ef79481 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a09147b8a98049f482b04933f45b69c7'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:39:59.931296 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1345/2718] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 24 17:39:59 2022] DELETE /identity/v3/users/a09147b8a98049f482b04933f45b69c7 => generated 0 bytes in 492 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:39:59.932673 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab5ceae-11a9-4c3c-bf7c-60ab0b6cb099 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:39:59.932964 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab5ceae-11a9-4c3c-bf7c-60ab0b6cb099 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:39:59.933244 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab5ceae-11a9-4c3c-bf7c-60ab0b6cb099 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:39:59.988901 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-8ab5ceae-11a9-4c3c-bf7c-60ab0b6cb099 None None] MFA Rules not processed for user `0e9ed4d47aef4140bfb2d3bb798cc559`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:39:59.990125 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-8ab5ceae-11a9-4c3c-bf7c-60ab0b6cb099 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:00.008389 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-b077ad9b-cc07-4ba0-8a04-2cf4f63082ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f9583f4a22f463695bee65c683b8163'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:00.023704 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-b077ad9b-cc07-4ba0-8a04-2cf4f63082ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f9583f4a22f463695bee65c683b8163'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:00.053906 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1346/2719] 10.222.0.18 () {58 vars in 1018 bytes} [Wed Aug 24 17:39:59 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:40:00.055030 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ef55c0-33a3-45a2-beca-fd0cbae52aa0 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:00.055318 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ef55c0-33a3-45a2-beca-fd0cbae52aa0 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:00.055586 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ef55c0-33a3-45a2-beca-fd0cbae52aa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:00.063611 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:40:00.079196 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-d3ef55c0-33a3-45a2-beca-fd0cbae52aa0 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:40:00.080477 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-d3ef55c0-33a3-45a2-beca-fd0cbae52aa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:00.097389 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-b077ad9b-cc07-4ba0-8a04-2cf4f63082ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f9583f4a22f463695bee65c683b8163'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:00.098628 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-b077ad9b-cc07-4ba0-8a04-2cf4f63082ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f9583f4a22f463695bee65c683b8163'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:00.114206 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-b077ad9b-cc07-4ba0-8a04-2cf4f63082ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f9583f4a22f463695bee65c683b8163'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:00.125651 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1374/2720] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 24 17:39:59 2022] DELETE /identity/v3/users/9f9583f4a22f463695bee65c683b8163 => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:40:00.125651 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b077ad9b-cc07-4ba0-8a04-2cf4f63082ba admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:00.126441 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-b077ad9b-cc07-4ba0-8a04-2cf4f63082ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:00.154084 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:40:00.157198 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1347/2721] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:40:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:40:00.167566 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ef55c0-33a3-45a2-beca-fd0cbae52aa0 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:00.170230 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-d3ef55c0-33a3-45a2-beca-fd0cbae52aa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:00.194031 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ef55c0-33a3-45a2-beca-fd0cbae52aa0 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:00.197950 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829226b0-715e-45c3-af43-b47af393b0ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e9ed4d47aef4140bfb2d3bb798cc559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:00.198583 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829226b0-715e-45c3-af43-b47af393b0ed service neutron] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:00.199062 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829226b0-715e-45c3-af43-b47af393b0ed service neutron] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:00.199341 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829226b0-715e-45c3-af43-b47af393b0ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:00.206737 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b077ad9b-cc07-4ba0-8a04-2cf4f63082ba admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:00.212990 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb09f1d5-4091-4e0c-b957-0fceab77b5f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e9ed4d47aef4140bfb2d3bb798cc559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:00.214939 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb09f1d5-4091-4e0c-b957-0fceab77b5f4 service neutron] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:00.214939 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb09f1d5-4091-4e0c-b957-0fceab77b5f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:00.214939 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb09f1d5-4091-4e0c-b957-0fceab77b5f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:00.216795 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-fb09f1d5-4091-4e0c-b957-0fceab77b5f4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:00.230495 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb09f1d5-4091-4e0c-b957-0fceab77b5f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:00.232202 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb09f1d5-4091-4e0c-b957-0fceab77b5f4 service neutron] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:00.242896 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1375/2722] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:40:00 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:40:00.276840 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829226b0-715e-45c3-af43-b47af393b0ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:00.278295 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829226b0-715e-45c3-af43-b47af393b0ed service neutron] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:00.290572 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1348/2723] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:40:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:40:00.354835 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219c90f1-17eb-4d1e-b8a2-e816602985db None None] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:00.355437 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219c90f1-17eb-4d1e-b8a2-e816602985db None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:00.355437 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219c90f1-17eb-4d1e-b8a2-e816602985db None None] PATH_INFO: `/` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:00.357864 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1376/2724] 10.222.0.18 () {54 vars in 921 bytes} [Wed Aug 24 17:40:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 17:40:00.363864 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e7bbbf-395b-4f70-99ec-5084e29d1f54 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:00.364061 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e7bbbf-395b-4f70-99ec-5084e29d1f54 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:00.364210 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e7bbbf-395b-4f70-99ec-5084e29d1f54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:00.412970 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-67e7bbbf-395b-4f70-99ec-5084e29d1f54 None None] MFA Rules not processed for user `557aca9255cb4bbb8af52e796c5c5177`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:40:00.414886 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-67e7bbbf-395b-4f70-99ec-5084e29d1f54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:00.497117 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1349/2725] 10.222.0.18 () {58 vars in 1018 bytes} [Wed Aug 24 17:40:00 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:40:00.508783 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219c90f1-17eb-4d1e-b8a2-e816602985db None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:00.510811 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-219c90f1-17eb-4d1e-b8a2-e816602985db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:00.526702 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219c90f1-17eb-4d1e-b8a2-e816602985db None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:00.531855 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e199c175-5a39-4efd-88d0-27b8c3b65842 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e9ed4d47aef4140bfb2d3bb798cc559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:00.532768 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e199c175-5a39-4efd-88d0-27b8c3b65842 service neutron] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:00.533203 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e199c175-5a39-4efd-88d0-27b8c3b65842 service neutron] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:00.533663 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e199c175-5a39-4efd-88d0-27b8c3b65842 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:00.568415 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e199c175-5a39-4efd-88d0-27b8c3b65842 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:00.571981 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e199c175-5a39-4efd-88d0-27b8c3b65842 service neutron] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:00.584591 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1377/2726] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:40:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:40:00.663537 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e7bbbf-395b-4f70-99ec-5084e29d1f54 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:00.665208 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-67e7bbbf-395b-4f70-99ec-5084e29d1f54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:00.694981 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e7bbbf-395b-4f70-99ec-5084e29d1f54 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:00.700162 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cd29c6-b893-4c89-b0e5-3e709679b970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:00.700909 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cd29c6-b893-4c89-b0e5-3e709679b970 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:00.701247 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cd29c6-b893-4c89-b0e5-3e709679b970 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:00.701589 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cd29c6-b893-4c89-b0e5-3e709679b970 admin admin] PATH_INFO: `/v3/projects/5842527515514eebaac1bffccc3561fc` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:00.702869 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cd29c6-b893-4c89-b0e5-3e709679b970 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5842527515514eebaac1bffccc3561fc)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:00.705190 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cd29c6-b893-4c89-b0e5-3e709679b970 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:00.717891 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:40:00.857326 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-70cd29c6-b893-4c89-b0e5-3e709679b970 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5842527515514eebaac1bffccc3561fc', 'request_id': 'req-70cd29c6-b893-4c89-b0e5-3e709679b970'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:00.935094 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-70cd29c6-b893-4c89-b0e5-3e709679b970 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5842527515514eebaac1bffccc3561fc', 'request_id': 'req-70cd29c6-b893-4c89-b0e5-3e709679b970'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:00.941905 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-70cd29c6-b893-4c89-b0e5-3e709679b970 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5842527515514eebaac1bffccc3561fc', 'request_id': 'req-70cd29c6-b893-4c89-b0e5-3e709679b970'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:00.952451 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-70cd29c6-b893-4c89-b0e5-3e709679b970 admin admin] The token cache is being invalidate because project 5842527515514eebaac1bffccc3561fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81732) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:40:00.952737 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-70cd29c6-b893-4c89-b0e5-3e709679b970 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:00.953505 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-70cd29c6-b893-4c89-b0e5-3e709679b970 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:00.955042 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1350/2727] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 24 17:40:00 2022] DELETE /identity/v3/projects/5842527515514eebaac1bffccc3561fc => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:40:00.959723 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e199c175-5a39-4efd-88d0-27b8c3b65842 service neutron] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:00.965913 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cd29c6-b893-4c89-b0e5-3e709679b970 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:00.967232 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-70cd29c6-b893-4c89-b0e5-3e709679b970 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:00.970863 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e199c175-5a39-4efd-88d0-27b8c3b65842 service neutron] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:01.039737 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc70353-2dbd-4e49-87d7-4f366bb83ff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:01.040381 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc70353-2dbd-4e49-87d7-4f366bb83ff2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:01.040694 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc70353-2dbd-4e49-87d7-4f366bb83ff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:01.041062 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc70353-2dbd-4e49-87d7-4f366bb83ff2 admin admin] PATH_INFO: `/v3/users/0bc30c70ea7941ada78ebc5db0045fa2` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:01.042504 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc70353-2dbd-4e49-87d7-4f366bb83ff2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0bc30c70ea7941ada78ebc5db0045fa2)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:01.044647 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc70353-2dbd-4e49-87d7-4f366bb83ff2 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:01.060433 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cd29c6-b893-4c89-b0e5-3e709679b970 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:01.065919 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100653e1-2013-4eb2-a2de-210dbeb37220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:01.066560 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100653e1-2013-4eb2-a2de-210dbeb37220 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:01.066840 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100653e1-2013-4eb2-a2de-210dbeb37220 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:01.067141 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100653e1-2013-4eb2-a2de-210dbeb37220 admin admin] PATH_INFO: `/v3/projects/c4d9fbf10f0e451d920dea6dd098224e` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:01.068306 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100653e1-2013-4eb2-a2de-210dbeb37220 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c4d9fbf10f0e451d920dea6dd098224e)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:01.070252 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100653e1-2013-4eb2-a2de-210dbeb37220 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:01.204049 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-ccc70353-2dbd-4e49-87d7-4f366bb83ff2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0bc30c70ea7941ada78ebc5db0045fa2'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:01.222871 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-ccc70353-2dbd-4e49-87d7-4f366bb83ff2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0bc30c70ea7941ada78ebc5db0045fa2'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:01.225830 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-100653e1-2013-4eb2-a2de-210dbeb37220 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c4d9fbf10f0e451d920dea6dd098224e', 'request_id': 'req-100653e1-2013-4eb2-a2de-210dbeb37220'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:01.311881 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-ccc70353-2dbd-4e49-87d7-4f366bb83ff2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0bc30c70ea7941ada78ebc5db0045fa2'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:01.312582 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-ccc70353-2dbd-4e49-87d7-4f366bb83ff2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0bc30c70ea7941ada78ebc5db0045fa2'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:01.323569 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-ccc70353-2dbd-4e49-87d7-4f366bb83ff2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0bc30c70ea7941ada78ebc5db0045fa2'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:01.330664 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1378/2728] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 24 17:40:00 2022] DELETE /identity/v3/users/0bc30c70ea7941ada78ebc5db0045fa2 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:40:01.392767 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-100653e1-2013-4eb2-a2de-210dbeb37220 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c4d9fbf10f0e451d920dea6dd098224e', 'request_id': 'req-100653e1-2013-4eb2-a2de-210dbeb37220'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:01.403977 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-100653e1-2013-4eb2-a2de-210dbeb37220 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c4d9fbf10f0e451d920dea6dd098224e', 'request_id': 'req-100653e1-2013-4eb2-a2de-210dbeb37220'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:01.420058 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-100653e1-2013-4eb2-a2de-210dbeb37220 admin admin] The token cache is being invalidate because project c4d9fbf10f0e451d920dea6dd098224e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81732) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:40:01.420344 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-100653e1-2013-4eb2-a2de-210dbeb37220 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:01.421025 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-100653e1-2013-4eb2-a2de-210dbeb37220 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:01.422684 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1351/2729] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 24 17:40:00 2022] DELETE /identity/v3/projects/c4d9fbf10f0e451d920dea6dd098224e => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:40:01.851323 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc70353-2dbd-4e49-87d7-4f366bb83ff2 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:01.852820 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-ccc70353-2dbd-4e49-87d7-4f366bb83ff2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:01.885242 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:40:01.928798 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc70353-2dbd-4e49-87d7-4f366bb83ff2 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:01.933609 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca778318-f673-456d-b30a-c7d785c513b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:01.934257 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca778318-f673-456d-b30a-c7d785c513b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:01.934529 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca778318-f673-456d-b30a-c7d785c513b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:01.935570 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca778318-f673-456d-b30a-c7d785c513b3 admin admin] PATH_INFO: `/v3/projects/4e66eb8c329647619eaeb94dc1de342e` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:01.936745 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca778318-f673-456d-b30a-c7d785c513b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e66eb8c329647619eaeb94dc1de342e)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:01.939385 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca778318-f673-456d-b30a-c7d785c513b3 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:02.068460 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-ca778318-f673-456d-b30a-c7d785c513b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e66eb8c329647619eaeb94dc1de342e', 'request_id': 'req-ca778318-f673-456d-b30a-c7d785c513b3'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:02.145776 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-ca778318-f673-456d-b30a-c7d785c513b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e66eb8c329647619eaeb94dc1de342e', 'request_id': 'req-ca778318-f673-456d-b30a-c7d785c513b3'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:02.153595 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-ca778318-f673-456d-b30a-c7d785c513b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e66eb8c329647619eaeb94dc1de342e', 'request_id': 'req-ca778318-f673-456d-b30a-c7d785c513b3'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:02.166515 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-ca778318-f673-456d-b30a-c7d785c513b3 admin admin] The token cache is being invalidate because project 4e66eb8c329647619eaeb94dc1de342e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81731) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:40:02.166781 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-ca778318-f673-456d-b30a-c7d785c513b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:02.167459 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-ca778318-f673-456d-b30a-c7d785c513b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:02.168956 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1379/2730] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 24 17:40:01 2022] DELETE /identity/v3/projects/4e66eb8c329647619eaeb94dc1de342e => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:40:02.178073 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6566b03-d85d-494a-9660-3d31ff4b1265 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:02.178586 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6566b03-d85d-494a-9660-3d31ff4b1265 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:02.179067 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6566b03-d85d-494a-9660-3d31ff4b1265 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:02.201492 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-f6566b03-d85d-494a-9660-3d31ff4b1265 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:40:02.202872 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-f6566b03-d85d-494a-9660-3d31ff4b1265 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:02.230966 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:40:02.273185 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1352/2731] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:40:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:40:02.284177 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376e8efa-6759-4d4c-a746-20e2ffa28040 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:02.284177 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376e8efa-6759-4d4c-a746-20e2ffa28040 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:02.284526 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376e8efa-6759-4d4c-a746-20e2ffa28040 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:02.306393 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-376e8efa-6759-4d4c-a746-20e2ffa28040 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:40:02.307645 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-376e8efa-6759-4d4c-a746-20e2ffa28040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:02.316422 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1380/2732] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:40:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:40:02.320467 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6566b03-d85d-494a-9660-3d31ff4b1265 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:02.330790 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6566b03-d85d-494a-9660-3d31ff4b1265 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:02.335932 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dafbbb-a435-4968-a916-ec03f110145b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:02.336627 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dafbbb-a435-4968-a916-ec03f110145b admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:02.336919 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dafbbb-a435-4968-a916-ec03f110145b admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:02.337196 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dafbbb-a435-4968-a916-ec03f110145b admin admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:02.338611 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dafbbb-a435-4968-a916-ec03f110145b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:40:02.338611 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dafbbb-a435-4968-a916-ec03f110145b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:02.339954 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dafbbb-a435-4968-a916-ec03f110145b admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:02.352901 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1353/2733] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 24 17:40:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:40:02.357373 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376e8efa-6759-4d4c-a746-20e2ffa28040 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:02.366853 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376e8efa-6759-4d4c-a746-20e2ffa28040 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:02.375573 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655816ef-7548-4554-915f-81c30bbcda9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:02.376194 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655816ef-7548-4554-915f-81c30bbcda9a admin admin] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:02.376475 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655816ef-7548-4554-915f-81c30bbcda9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:02.376750 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655816ef-7548-4554-915f-81c30bbcda9a admin admin] PATH_INFO: `/v3/projects` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:02.377382 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655816ef-7548-4554-915f-81c30bbcda9a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:02.379340 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655816ef-7548-4554-915f-81c30bbcda9a admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:02.451828 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1381/2734] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 24 17:40:02 2022] POST /identity/v3/projects => generated 373 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:40:02.456932 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dafbbb-a435-4968-a916-ec03f110145b admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:02.468765 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dafbbb-a435-4968-a916-ec03f110145b admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:02.534361 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd6a7a7-265b-42d8-a207-0d459d1a14f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:02.535014 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd6a7a7-265b-42d8-a207-0d459d1a14f7 admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:02.535301 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd6a7a7-265b-42d8-a207-0d459d1a14f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:02.535577 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd6a7a7-265b-42d8-a207-0d459d1a14f7 admin admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:02.536203 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd6a7a7-265b-42d8-a207-0d459d1a14f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:02.538588 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd6a7a7-265b-42d8-a207-0d459d1a14f7 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:02.640690 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1354/2735] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 24 17:40:02 2022] POST /identity/v3/users => generated 287 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:40:02.644983 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655816ef-7548-4554-915f-81c30bbcda9a admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:02.656079 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655816ef-7548-4554-915f-81c30bbcda9a admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:02.660921 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a333ba0-3f47-4343-b7dd-af54552cacaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:02.661542 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a333ba0-3f47-4343-b7dd-af54552cacaa admin admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:02.661823 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a333ba0-3f47-4343-b7dd-af54552cacaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:02.662097 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a333ba0-3f47-4343-b7dd-af54552cacaa admin admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:02.662748 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a333ba0-3f47-4343-b7dd-af54552cacaa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:02.664513 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a333ba0-3f47-4343-b7dd-af54552cacaa admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:02.678872 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1382/2736] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:40:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:40:02.681285 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd6a7a7-265b-42d8-a207-0d459d1a14f7 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:02.694395 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd6a7a7-265b-42d8-a207-0d459d1a14f7 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:02.700318 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e3fa61-1583-4436-89e4-4828fc81f74b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:02.707287 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e3fa61-1583-4436-89e4-4828fc81f74b admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:02.707287 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e3fa61-1583-4436-89e4-4828fc81f74b admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:02.707287 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e3fa61-1583-4436-89e4-4828fc81f74b admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:02.707287 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e3fa61-1583-4436-89e4-4828fc81f74b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:02.707287 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e3fa61-1583-4436-89e4-4828fc81f74b admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:02.716289 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1355/2737] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:40:02 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:40:02.720777 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a333ba0-3f47-4343-b7dd-af54552cacaa admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:02.731097 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a333ba0-3f47-4343-b7dd-af54552cacaa admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:02.736393 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af129c9-77c3-4d75-ad04-b2f596fac645 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:02.737069 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af129c9-77c3-4d75-ad04-b2f596fac645 admin admin] REQUEST_METHOD: `PUT` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:02.737354 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af129c9-77c3-4d75-ad04-b2f596fac645 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:02.737634 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af129c9-77c3-4d75-ad04-b2f596fac645 admin admin] PATH_INFO: `/v3/projects/eefa26ea0f6f421899f307bb70d6376d/users/e7670e6eedde44c9ab9ffd0d63d4be49/roles/60dc7e16aee746d5a3e8aa27d7d8525c` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:02.753318 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af129c9-77c3-4d75-ad04-b2f596fac645 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eefa26ea0f6f421899f307bb70d6376d, user_id=e7670e6eedde44c9ab9ffd0d63d4be49, role_id=60dc7e16aee746d5a3e8aa27d7d8525c)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:02.756048 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af129c9-77c3-4d75-ad04-b2f596fac645 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:02.832796 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1383/2738] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 24 17:40:02 2022] PUT /identity/v3/projects/eefa26ea0f6f421899f307bb70d6376d/users/e7670e6eedde44c9ab9ffd0d63d4be49/roles/60dc7e16aee746d5a3e8aa27d7d8525c => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:40:02.844509 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4535330-ceba-4a01-8f25-9bd42ad154c3 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:02.844793 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4535330-ceba-4a01-8f25-9bd42ad154c3 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:02.845063 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4535330-ceba-4a01-8f25-9bd42ad154c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:02.883260 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-c4535330-ceba-4a01-8f25-9bd42ad154c3 None None] MFA Rules not processed for user `e7670e6eedde44c9ab9ffd0d63d4be49`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:40:02.885828 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-c4535330-ceba-4a01-8f25-9bd42ad154c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:02.972946 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1356/2739] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:40:02 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:40:02.984974 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af129c9-77c3-4d75-ad04-b2f596fac645 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:02.987910 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-1af129c9-77c3-4d75-ad04-b2f596fac645 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:03.051326 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af129c9-77c3-4d75-ad04-b2f596fac645 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:03.055071 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3718e0fc-13e3-4418-80de-b1e1f4d7733c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be06246d47584b529150ceb98a5cfd95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:03.055685 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3718e0fc-13e3-4418-80de-b1e1f4d7733c service cinder] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:03.055968 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3718e0fc-13e3-4418-80de-b1e1f4d7733c service cinder] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:03.056240 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3718e0fc-13e3-4418-80de-b1e1f4d7733c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:03.061615 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:40:03.065969 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3718e0fc-13e3-4418-80de-b1e1f4d7733c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:03.067467 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3718e0fc-13e3-4418-80de-b1e1f4d7733c service cinder] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:03.077339 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1384/2740] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:40:02 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:40:03.104810 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4535330-ceba-4a01-8f25-9bd42ad154c3 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:03.140175 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:40:03.185031 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4535330-ceba-4a01-8f25-9bd42ad154c3 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:03.190991 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c92f7d0-ba2e-41ea-b68b-90b61a7e3062 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:03.191728 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c92f7d0-ba2e-41ea-b68b-90b61a7e3062 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:03.191949 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c92f7d0-ba2e-41ea-b68b-90b61a7e3062 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:03.192265 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c92f7d0-ba2e-41ea-b68b-90b61a7e3062 admin admin] PATH_INFO: `/v3/users/e7670e6eedde44c9ab9ffd0d63d4be49` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:03.193455 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c92f7d0-ba2e-41ea-b68b-90b61a7e3062 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7670e6eedde44c9ab9ffd0d63d4be49)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:03.195486 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c92f7d0-ba2e-41ea-b68b-90b61a7e3062 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:03.395530 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-5c92f7d0-ba2e-41ea-b68b-90b61a7e3062 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7670e6eedde44c9ab9ffd0d63d4be49'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:03.491499 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-5c92f7d0-ba2e-41ea-b68b-90b61a7e3062 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7670e6eedde44c9ab9ffd0d63d4be49'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:03.492253 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-5c92f7d0-ba2e-41ea-b68b-90b61a7e3062 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7670e6eedde44c9ab9ffd0d63d4be49'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:03.502377 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-5c92f7d0-ba2e-41ea-b68b-90b61a7e3062 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7670e6eedde44c9ab9ffd0d63d4be49'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:03.503443 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-5c92f7d0-ba2e-41ea-b68b-90b61a7e3062 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7670e6eedde44c9ab9ffd0d63d4be49'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:03.531440 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1357/2741] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 24 17:40:03 2022] DELETE /identity/v3/users/e7670e6eedde44c9ab9ffd0d63d4be49 => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:40:03.537430 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb9a243-7e51-498c-8a91-80999e104524 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:03.537720 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb9a243-7e51-498c-8a91-80999e104524 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:03.538044 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb9a243-7e51-498c-8a91-80999e104524 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:03.569620 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-0eb9a243-7e51-498c-8a91-80999e104524 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:40:03.571624 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-0eb9a243-7e51-498c-8a91-80999e104524 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:03.786884 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1385/2742] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:40:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:40:03.797563 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c92f7d0-ba2e-41ea-b68b-90b61a7e3062 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:03.799716 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-5c92f7d0-ba2e-41ea-b68b-90b61a7e3062 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:03.867317 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c92f7d0-ba2e-41ea-b68b-90b61a7e3062 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:03.871367 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f432208-fe73-4ebb-92be-297d2146e307 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e9ed4d47aef4140bfb2d3bb798cc559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:03.871994 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f432208-fe73-4ebb-92be-297d2146e307 service neutron] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:03.872297 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f432208-fe73-4ebb-92be-297d2146e307 service neutron] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:03.872564 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f432208-fe73-4ebb-92be-297d2146e307 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:03.884254 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f432208-fe73-4ebb-92be-297d2146e307 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:03.885723 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f432208-fe73-4ebb-92be-297d2146e307 service neutron] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:03.897712 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1358/2743] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:40:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:40:04.560233 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb9a243-7e51-498c-8a91-80999e104524 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:04.561979 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-0eb9a243-7e51-498c-8a91-80999e104524 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:04.575551 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb9a243-7e51-498c-8a91-80999e104524 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:04.581996 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f88022-815a-40c0-a0f5-70877972cad9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:04.587207 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f88022-815a-40c0-a0f5-70877972cad9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:04.587207 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f88022-815a-40c0-a0f5-70877972cad9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:04.587207 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f88022-815a-40c0-a0f5-70877972cad9 admin admin] PATH_INFO: `/v3/projects/eefa26ea0f6f421899f307bb70d6376d` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:04.587207 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f88022-815a-40c0-a0f5-70877972cad9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eefa26ea0f6f421899f307bb70d6376d)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:04.587207 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f88022-815a-40c0-a0f5-70877972cad9 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:04.774805 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-e8f88022-815a-40c0-a0f5-70877972cad9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eefa26ea0f6f421899f307bb70d6376d', 'request_id': 'req-e8f88022-815a-40c0-a0f5-70877972cad9'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:04.866775 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-e8f88022-815a-40c0-a0f5-70877972cad9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eefa26ea0f6f421899f307bb70d6376d', 'request_id': 'req-e8f88022-815a-40c0-a0f5-70877972cad9'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:04.887218 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-e8f88022-815a-40c0-a0f5-70877972cad9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eefa26ea0f6f421899f307bb70d6376d', 'request_id': 'req-e8f88022-815a-40c0-a0f5-70877972cad9'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:04.899861 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-e8f88022-815a-40c0-a0f5-70877972cad9 admin admin] The token cache is being invalidate because project eefa26ea0f6f421899f307bb70d6376d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81731) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:40:04.900154 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-e8f88022-815a-40c0-a0f5-70877972cad9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:04.900928 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-e8f88022-815a-40c0-a0f5-70877972cad9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:04.902411 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1386/2744] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 24 17:40:04 2022] DELETE /identity/v3/projects/eefa26ea0f6f421899f307bb70d6376d => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:40:04.908401 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a087f446-6794-414f-ade0-2df8f3350d13 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:04.908976 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a087f446-6794-414f-ade0-2df8f3350d13 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:04.908976 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a087f446-6794-414f-ade0-2df8f3350d13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:04.930158 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-a087f446-6794-414f-ade0-2df8f3350d13 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:40:04.931455 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-a087f446-6794-414f-ade0-2df8f3350d13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:04.996063 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:40:05.027698 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1359/2745] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:40:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:40:05.036497 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f84bc71-9717-49b8-93c9-dc8b37e71c89 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:05.036779 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f84bc71-9717-49b8-93c9-dc8b37e71c89 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:05.037050 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f84bc71-9717-49b8-93c9-dc8b37e71c89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:05.057973 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-1f84bc71-9717-49b8-93c9-dc8b37e71c89 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:40:05.059272 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-1f84bc71-9717-49b8-93c9-dc8b37e71c89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:05.069257 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1387/2746] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:40:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:40:05.074190 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a087f446-6794-414f-ade0-2df8f3350d13 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:05.085588 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a087f446-6794-414f-ade0-2df8f3350d13 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:05.092069 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb2b42a-7777-4310-af4a-5b8dd725ebf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:05.092728 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb2b42a-7777-4310-af4a-5b8dd725ebf7 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:05.093004 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb2b42a-7777-4310-af4a-5b8dd725ebf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:05.093386 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb2b42a-7777-4310-af4a-5b8dd725ebf7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:05.094732 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb2b42a-7777-4310-af4a-5b8dd725ebf7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:40:05.095081 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb2b42a-7777-4310-af4a-5b8dd725ebf7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:05.096794 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb2b42a-7777-4310-af4a-5b8dd725ebf7 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:05.114505 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1360/2747] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 24 17:40:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:40:05.116045 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f84bc71-9717-49b8-93c9-dc8b37e71c89 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:05.125810 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f84bc71-9717-49b8-93c9-dc8b37e71c89 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:05.133955 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc54ba3e-3bff-4d91-9030-eadc28545a97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:05.134605 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc54ba3e-3bff-4d91-9030-eadc28545a97 admin admin] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:05.134936 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc54ba3e-3bff-4d91-9030-eadc28545a97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:05.135203 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc54ba3e-3bff-4d91-9030-eadc28545a97 admin admin] PATH_INFO: `/v3/projects` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:05.135894 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc54ba3e-3bff-4d91-9030-eadc28545a97 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:05.137845 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc54ba3e-3bff-4d91-9030-eadc28545a97 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:05.252717 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1388/2748] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 24 17:40:05 2022] POST /identity/v3/projects => generated 375 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:40:05.256973 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb2b42a-7777-4310-af4a-5b8dd725ebf7 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:05.272801 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb2b42a-7777-4310-af4a-5b8dd725ebf7 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:05.332805 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c921fe6-c830-41c1-ad7c-e5764427df9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:05.333426 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c921fe6-c830-41c1-ad7c-e5764427df9f admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:05.333711 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c921fe6-c830-41c1-ad7c-e5764427df9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:05.333987 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c921fe6-c830-41c1-ad7c-e5764427df9f admin admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:05.334616 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c921fe6-c830-41c1-ad7c-e5764427df9f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:05.336599 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c921fe6-c830-41c1-ad7c-e5764427df9f admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:05.436290 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1361/2749] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 24 17:40:05 2022] POST /identity/v3/users => generated 288 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:40:05.438395 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc54ba3e-3bff-4d91-9030-eadc28545a97 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:05.450553 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc54ba3e-3bff-4d91-9030-eadc28545a97 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:05.456598 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d871bedb-91d5-4a39-9075-70d4cc6ff1dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:05.456789 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d871bedb-91d5-4a39-9075-70d4cc6ff1dd admin admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:05.457133 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d871bedb-91d5-4a39-9075-70d4cc6ff1dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:05.457514 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d871bedb-91d5-4a39-9075-70d4cc6ff1dd admin admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:05.458190 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d871bedb-91d5-4a39-9075-70d4cc6ff1dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:05.460032 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d871bedb-91d5-4a39-9075-70d4cc6ff1dd admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:05.473679 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1389/2750] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:40:05 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:40:05.475219 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c921fe6-c830-41c1-ad7c-e5764427df9f admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:05.486100 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c921fe6-c830-41c1-ad7c-e5764427df9f admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:05.491124 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75631d6-63c1-41e7-acc2-5685495a47ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:05.491957 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75631d6-63c1-41e7-acc2-5685495a47ff admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:05.492452 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75631d6-63c1-41e7-acc2-5685495a47ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:05.492735 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75631d6-63c1-41e7-acc2-5685495a47ff admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:05.493392 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75631d6-63c1-41e7-acc2-5685495a47ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:05.495100 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75631d6-63c1-41e7-acc2-5685495a47ff admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:05.508167 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1362/2751] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:40:05 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:40:05.511657 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d871bedb-91d5-4a39-9075-70d4cc6ff1dd admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:05.522695 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d871bedb-91d5-4a39-9075-70d4cc6ff1dd admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:05.528306 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46eff898-1af8-4e5c-8bd8-ddec967f19e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:05.528980 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eff898-1af8-4e5c-8bd8-ddec967f19e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:05.529319 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eff898-1af8-4e5c-8bd8-ddec967f19e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:05.529593 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eff898-1af8-4e5c-8bd8-ddec967f19e9 admin admin] PATH_INFO: `/v3/projects/71b79a5db107412f97be56c247d337f1/users/8d4898a6a5b2417d91d97eb33be44cf8/roles/60dc7e16aee746d5a3e8aa27d7d8525c` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:05.558772 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46eff898-1af8-4e5c-8bd8-ddec967f19e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=71b79a5db107412f97be56c247d337f1, user_id=8d4898a6a5b2417d91d97eb33be44cf8, role_id=60dc7e16aee746d5a3e8aa27d7d8525c)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:05.561601 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46eff898-1af8-4e5c-8bd8-ddec967f19e9 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:05.639066 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1390/2752] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 24 17:40:05 2022] PUT /identity/v3/projects/71b79a5db107412f97be56c247d337f1/users/8d4898a6a5b2417d91d97eb33be44cf8/roles/60dc7e16aee746d5a3e8aa27d7d8525c => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:40:05.650610 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f32ea0-a433-461f-9d1e-c087383dbd6b None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:05.651061 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f32ea0-a433-461f-9d1e-c087383dbd6b None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:05.651234 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f32ea0-a433-461f-9d1e-c087383dbd6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:05.716849 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-63f32ea0-a433-461f-9d1e-c087383dbd6b None None] MFA Rules not processed for user `8d4898a6a5b2417d91d97eb33be44cf8`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:40:05.719314 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-63f32ea0-a433-461f-9d1e-c087383dbd6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:05.860912 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1363/2753] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:40:05 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:40:05.868452 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46eff898-1af8-4e5c-8bd8-ddec967f19e9 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:05.870117 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-46eff898-1af8-4e5c-8bd8-ddec967f19e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:05.893272 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:40:05.948650 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46eff898-1af8-4e5c-8bd8-ddec967f19e9 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:05.952902 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7f653f-edbd-4edf-89e5-a3caabf4f57c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be06246d47584b529150ceb98a5cfd95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:05.954216 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7f653f-edbd-4edf-89e5-a3caabf4f57c service cinder] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:05.954400 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7f653f-edbd-4edf-89e5-a3caabf4f57c service cinder] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:05.954676 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7f653f-edbd-4edf-89e5-a3caabf4f57c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:05.963216 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7f653f-edbd-4edf-89e5-a3caabf4f57c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:05.964655 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7f653f-edbd-4edf-89e5-a3caabf4f57c service cinder] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:05.974012 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1391/2754] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:40:05 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:40:14.175675 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f32ea0-a433-461f-9d1e-c087383dbd6b None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:14.177353 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-63f32ea0-a433-461f-9d1e-c087383dbd6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:14.225421 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:40:14.237054 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f32ea0-a433-461f-9d1e-c087383dbd6b None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:14.240653 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4759da77-951f-43fa-b16b-adaf10f53d9c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e9ed4d47aef4140bfb2d3bb798cc559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:14.241258 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4759da77-951f-43fa-b16b-adaf10f53d9c service neutron] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:14.241572 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4759da77-951f-43fa-b16b-adaf10f53d9c service neutron] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:14.241886 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4759da77-951f-43fa-b16b-adaf10f53d9c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:14.243584 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-4759da77-951f-43fa-b16b-adaf10f53d9c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:14.308093 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4759da77-951f-43fa-b16b-adaf10f53d9c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:14.309548 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4759da77-951f-43fa-b16b-adaf10f53d9c service neutron] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:14.320624 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1364/2755] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:40:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:40:16.089381 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=20 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:40:18.199828 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7f653f-edbd-4edf-89e5-a3caabf4f57c service cinder] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:18.201397 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-4e7f653f-edbd-4edf-89e5-a3caabf4f57c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:18.273355 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7f653f-edbd-4edf-89e5-a3caabf4f57c service cinder] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:18.278258 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4431217-416c-4c67-bd93-f9a590e86e30 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03fc77c6107e430f90f7b6f7cfd4ec02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:18.278859 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4431217-416c-4c67-bd93-f9a590e86e30 service placement] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:18.279179 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4431217-416c-4c67-bd93-f9a590e86e30 service placement] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:18.279452 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4431217-416c-4c67-bd93-f9a590e86e30 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:18.281595 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-f4431217-416c-4c67-bd93-f9a590e86e30 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:18.293730 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4431217-416c-4c67-bd93-f9a590e86e30 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:18.295341 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4431217-416c-4c67-bd93-f9a590e86e30 service placement] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:18.306710 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1392/2756] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:40:18 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:40:18.330471 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4759da77-951f-43fa-b16b-adaf10f53d9c service neutron] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:18.331979 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-4759da77-951f-43fa-b16b-adaf10f53d9c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:18.348717 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4759da77-951f-43fa-b16b-adaf10f53d9c service neutron] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:18.353877 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b461b433-350f-4b20-9846-94a9dc72352b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:18.354516 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b461b433-350f-4b20-9846-94a9dc72352b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:18.354803 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b461b433-350f-4b20-9846-94a9dc72352b admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:18.355108 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b461b433-350f-4b20-9846-94a9dc72352b admin admin] PATH_INFO: `/v3/users/160ce86c89ea449da346d98d20ce2a9e` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:18.356218 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b461b433-350f-4b20-9846-94a9dc72352b admin admin] RBAC: Authorizing `identity:delete_user(user_id=160ce86c89ea449da346d98d20ce2a9e)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:18.358121 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b461b433-350f-4b20-9846-94a9dc72352b admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:18.601322 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-b461b433-350f-4b20-9846-94a9dc72352b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '160ce86c89ea449da346d98d20ce2a9e'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:18.683400 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-b461b433-350f-4b20-9846-94a9dc72352b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '160ce86c89ea449da346d98d20ce2a9e'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:18.684066 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-b461b433-350f-4b20-9846-94a9dc72352b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '160ce86c89ea449da346d98d20ce2a9e'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:18.695111 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-b461b433-350f-4b20-9846-94a9dc72352b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '160ce86c89ea449da346d98d20ce2a9e'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:18.695111 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-b461b433-350f-4b20-9846-94a9dc72352b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '160ce86c89ea449da346d98d20ce2a9e'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:18.716117 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1365/2757] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 24 17:40:18 2022] DELETE /identity/v3/users/160ce86c89ea449da346d98d20ce2a9e => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:40:19.000425 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4431217-416c-4c67-bd93-f9a590e86e30 service placement] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:19.014340 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4431217-416c-4c67-bd93-f9a590e86e30 service placement] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:19.084484 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f747a76c-df1f-46ee-803f-7c1ba70855f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:19.085035 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f747a76c-df1f-46ee-803f-7c1ba70855f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:19.085314 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f747a76c-df1f-46ee-803f-7c1ba70855f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:19.085578 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f747a76c-df1f-46ee-803f-7c1ba70855f1 admin admin] PATH_INFO: `/v3/projects/f6ebea743b3843f3b6274ddfc0b5b73a` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:19.086696 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f747a76c-df1f-46ee-803f-7c1ba70855f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6ebea743b3843f3b6274ddfc0b5b73a)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:19.088678 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f747a76c-df1f-46ee-803f-7c1ba70855f1 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:19.192541 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-f747a76c-df1f-46ee-803f-7c1ba70855f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6ebea743b3843f3b6274ddfc0b5b73a', 'request_id': 'req-f747a76c-df1f-46ee-803f-7c1ba70855f1'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:19.260952 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-f747a76c-df1f-46ee-803f-7c1ba70855f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6ebea743b3843f3b6274ddfc0b5b73a', 'request_id': 'req-f747a76c-df1f-46ee-803f-7c1ba70855f1'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:19.269124 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-f747a76c-df1f-46ee-803f-7c1ba70855f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6ebea743b3843f3b6274ddfc0b5b73a', 'request_id': 'req-f747a76c-df1f-46ee-803f-7c1ba70855f1'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:19.280553 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-f747a76c-df1f-46ee-803f-7c1ba70855f1 admin admin] The token cache is being invalidate because project f6ebea743b3843f3b6274ddfc0b5b73a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81731) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:40:19.281854 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-f747a76c-df1f-46ee-803f-7c1ba70855f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:19.282667 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-f747a76c-df1f-46ee-803f-7c1ba70855f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:19.284538 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1393/2758] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 24 17:40:18 2022] DELETE /identity/v3/projects/f6ebea743b3843f3b6274ddfc0b5b73a => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:40:19.291206 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49346fca-4eef-45bd-88e0-a10655e97003 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:19.291594 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49346fca-4eef-45bd-88e0-a10655e97003 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:19.291926 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49346fca-4eef-45bd-88e0-a10655e97003 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:19.312209 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-49346fca-4eef-45bd-88e0-a10655e97003 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:40:19.313413 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-49346fca-4eef-45bd-88e0-a10655e97003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:19.376285 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1366/2759] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:40:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:40:19.384478 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d279ec7e-e31e-4e18-af96-b3d95865eaf6 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:19.384757 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d279ec7e-e31e-4e18-af96-b3d95865eaf6 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:19.385031 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d279ec7e-e31e-4e18-af96-b3d95865eaf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:19.406877 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-d279ec7e-e31e-4e18-af96-b3d95865eaf6 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:40:19.408189 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-d279ec7e-e31e-4e18-af96-b3d95865eaf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:19.416168 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1394/2760] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:40:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:40:19.420426 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49346fca-4eef-45bd-88e0-a10655e97003 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:19.426468 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:40:19.430187 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49346fca-4eef-45bd-88e0-a10655e97003 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:19.435136 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030c6154-8f43-4909-a2d0-a84c1779d919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:19.437679 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030c6154-8f43-4909-a2d0-a84c1779d919 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:19.437679 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030c6154-8f43-4909-a2d0-a84c1779d919 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:19.437679 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030c6154-8f43-4909-a2d0-a84c1779d919 admin admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:19.437679 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030c6154-8f43-4909-a2d0-a84c1779d919 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:40:19.438072 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030c6154-8f43-4909-a2d0-a84c1779d919 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:19.439343 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030c6154-8f43-4909-a2d0-a84c1779d919 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:19.451315 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1367/2761] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 24 17:40:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:40:19.456198 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d279ec7e-e31e-4e18-af96-b3d95865eaf6 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:19.465465 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d279ec7e-e31e-4e18-af96-b3d95865eaf6 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:19.470165 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee35acb9-d9c7-4879-8671-f6e8e6188535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:19.470766 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee35acb9-d9c7-4879-8671-f6e8e6188535 admin admin] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:19.471084 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee35acb9-d9c7-4879-8671-f6e8e6188535 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:19.471360 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee35acb9-d9c7-4879-8671-f6e8e6188535 admin admin] PATH_INFO: `/v3/projects` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:19.471994 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee35acb9-d9c7-4879-8671-f6e8e6188535 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:19.473863 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee35acb9-d9c7-4879-8671-f6e8e6188535 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:19.485549 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:40:19.561701 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030c6154-8f43-4909-a2d0-a84c1779d919 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:19.562429 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1395/2762] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 24 17:40:19 2022] POST /identity/v3/projects => generated 365 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:40:19.572504 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030c6154-8f43-4909-a2d0-a84c1779d919 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:19.638659 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa6d414-6fba-4416-a01d-9506827b7ef2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:19.639220 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa6d414-6fba-4416-a01d-9506827b7ef2 admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:19.639504 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa6d414-6fba-4416-a01d-9506827b7ef2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:19.639779 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa6d414-6fba-4416-a01d-9506827b7ef2 admin admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:19.640973 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa6d414-6fba-4416-a01d-9506827b7ef2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:19.643365 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa6d414-6fba-4416-a01d-9506827b7ef2 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:19.741637 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1368/2763] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 24 17:40:19 2022] POST /identity/v3/users => generated 283 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:40:19.745449 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee35acb9-d9c7-4879-8671-f6e8e6188535 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:19.756201 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee35acb9-d9c7-4879-8671-f6e8e6188535 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:19.761041 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0746aefd-86a6-4dc5-9c36-87ace34fc43f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:19.761765 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0746aefd-86a6-4dc5-9c36-87ace34fc43f admin admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:19.762201 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0746aefd-86a6-4dc5-9c36-87ace34fc43f admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:19.762525 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0746aefd-86a6-4dc5-9c36-87ace34fc43f admin admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:19.763196 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0746aefd-86a6-4dc5-9c36-87ace34fc43f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:19.768118 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0746aefd-86a6-4dc5-9c36-87ace34fc43f admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:19.775727 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1396/2764] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:40:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:40:19.779683 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa6d414-6fba-4416-a01d-9506827b7ef2 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:19.794173 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa6d414-6fba-4416-a01d-9506827b7ef2 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:19.801560 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d1351c-e49a-42d6-98db-23d8debc0d85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:19.802465 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d1351c-e49a-42d6-98db-23d8debc0d85 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:19.802965 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d1351c-e49a-42d6-98db-23d8debc0d85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:19.803430 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d1351c-e49a-42d6-98db-23d8debc0d85 admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:19.804443 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d1351c-e49a-42d6-98db-23d8debc0d85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:19.806809 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d1351c-e49a-42d6-98db-23d8debc0d85 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:19.823647 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1369/2765] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:40:19 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:40:19.827891 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0746aefd-86a6-4dc5-9c36-87ace34fc43f admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:19.838260 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0746aefd-86a6-4dc5-9c36-87ace34fc43f admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:19.843070 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf3be9a-228f-4c2f-bafc-6f623cd6a392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:19.843909 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf3be9a-228f-4c2f-bafc-6f623cd6a392 admin admin] REQUEST_METHOD: `PUT` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:19.844197 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf3be9a-228f-4c2f-bafc-6f623cd6a392 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:19.844475 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf3be9a-228f-4c2f-bafc-6f623cd6a392 admin admin] PATH_INFO: `/v3/projects/46e54d0ba39c4f2099007a429bc66881/users/349d5cbe53d342d5b4bf82752a4c4bdf/roles/60dc7e16aee746d5a3e8aa27d7d8525c` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:19.860661 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf3be9a-228f-4c2f-bafc-6f623cd6a392 admin admin] RBAC: Authorizing `identity:create_grant(project_id=46e54d0ba39c4f2099007a429bc66881, user_id=349d5cbe53d342d5b4bf82752a4c4bdf, role_id=60dc7e16aee746d5a3e8aa27d7d8525c)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:19.864970 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf3be9a-228f-4c2f-bafc-6f623cd6a392 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:20.026592 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1397/2766] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 24 17:40:19 2022] PUT /identity/v3/projects/46e54d0ba39c4f2099007a429bc66881/users/349d5cbe53d342d5b4bf82752a4c4bdf/roles/60dc7e16aee746d5a3e8aa27d7d8525c => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:40:20.039344 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde84c7e-7dd0-45f1-85f6-513c5bb40728 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:20.039628 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde84c7e-7dd0-45f1-85f6-513c5bb40728 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:20.039901 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde84c7e-7dd0-45f1-85f6-513c5bb40728 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:20.069472 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-dde84c7e-7dd0-45f1-85f6-513c5bb40728 None None] MFA Rules not processed for user `349d5cbe53d342d5b4bf82752a4c4bdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:40:20.070690 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-dde84c7e-7dd0-45f1-85f6-513c5bb40728 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:20.137021 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1370/2767] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:40:20 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:40:20.148219 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf3be9a-228f-4c2f-bafc-6f623cd6a392 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:20.149814 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-acf3be9a-228f-4c2f-bafc-6f623cd6a392 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:20.214998 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf3be9a-228f-4c2f-bafc-6f623cd6a392 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:20.220264 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8a86d7-fedf-4dde-877c-3364bc6f82bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be06246d47584b529150ceb98a5cfd95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:20.220892 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8a86d7-fedf-4dde-877c-3364bc6f82bd service cinder] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:20.221173 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8a86d7-fedf-4dde-877c-3364bc6f82bd service cinder] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:20.221442 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8a86d7-fedf-4dde-877c-3364bc6f82bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:20.230679 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8a86d7-fedf-4dde-877c-3364bc6f82bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:20.232154 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8a86d7-fedf-4dde-877c-3364bc6f82bd service cinder] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:20.242410 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1398/2768] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:40:20 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:40:23.770874 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde84c7e-7dd0-45f1-85f6-513c5bb40728 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:23.772498 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-dde84c7e-7dd0-45f1-85f6-513c5bb40728 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:23.841792 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:40:23.859415 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde84c7e-7dd0-45f1-85f6-513c5bb40728 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:23.864815 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9fd874-268a-4f64-afaf-120fd362e310 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557aca9255cb4bbb8af52e796c5c5177', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:23.865446 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9fd874-268a-4f64-afaf-120fd362e310 service nova] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:23.865724 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9fd874-268a-4f64-afaf-120fd362e310 service nova] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:23.865993 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9fd874-268a-4f64-afaf-120fd362e310 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:23.868137 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-ae9fd874-268a-4f64-afaf-120fd362e310 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:23.926876 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9fd874-268a-4f64-afaf-120fd362e310 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:23.928612 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9fd874-268a-4f64-afaf-120fd362e310 service nova] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:23.939427 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1371/2769] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:40:23 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:40:30.238056 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=44,keystone:INSERT=2 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:40:33.355161 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8a86d7-fedf-4dde-877c-3364bc6f82bd service cinder] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:33.422237 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8a86d7-fedf-4dde-877c-3364bc6f82bd service cinder] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:33.427026 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d01dda-e4a0-4336-ab7b-efee46b11b2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:33.427654 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d01dda-e4a0-4336-ab7b-efee46b11b2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:33.427934 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d01dda-e4a0-4336-ab7b-efee46b11b2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:33.428207 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d01dda-e4a0-4336-ab7b-efee46b11b2c admin admin] PATH_INFO: `/v3/users/349d5cbe53d342d5b4bf82752a4c4bdf` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:33.429317 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d01dda-e4a0-4336-ab7b-efee46b11b2c admin admin] RBAC: Authorizing `identity:delete_user(user_id=349d5cbe53d342d5b4bf82752a4c4bdf)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:33.431334 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d01dda-e4a0-4336-ab7b-efee46b11b2c admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:33.606181 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-65d01dda-e4a0-4336-ab7b-efee46b11b2c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '349d5cbe53d342d5b4bf82752a4c4bdf'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:33.618369 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-65d01dda-e4a0-4336-ab7b-efee46b11b2c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '349d5cbe53d342d5b4bf82752a4c4bdf'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:33.671736 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-65d01dda-e4a0-4336-ab7b-efee46b11b2c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '349d5cbe53d342d5b4bf82752a4c4bdf'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:33.672427 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-65d01dda-e4a0-4336-ab7b-efee46b11b2c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '349d5cbe53d342d5b4bf82752a4c4bdf'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:33.680162 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-65d01dda-e4a0-4336-ab7b-efee46b11b2c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '349d5cbe53d342d5b4bf82752a4c4bdf'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:33.686315 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1399/2770] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 24 17:40:33 2022] DELETE /identity/v3/users/349d5cbe53d342d5b4bf82752a4c4bdf => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:40:33.691739 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179682fe-c481-4364-af33-f624b1963211 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:33.692224 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179682fe-c481-4364-af33-f624b1963211 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:33.692348 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179682fe-c481-4364-af33-f624b1963211 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:33.711955 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-179682fe-c481-4364-af33-f624b1963211 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:40:33.713163 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-179682fe-c481-4364-af33-f624b1963211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:33.781584 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1372/2771] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:40:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:40:33.793061 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d01dda-e4a0-4336-ab7b-efee46b11b2c admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:33.794469 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-65d01dda-e4a0-4336-ab7b-efee46b11b2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:33.854475 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d01dda-e4a0-4336-ab7b-efee46b11b2c admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:33.858797 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd71cd09-c455-4f40-b4d9-b79058bf87a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e9ed4d47aef4140bfb2d3bb798cc559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:33.859974 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd71cd09-c455-4f40-b4d9-b79058bf87a8 service neutron] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:33.860476 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd71cd09-c455-4f40-b4d9-b79058bf87a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:33.860944 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd71cd09-c455-4f40-b4d9-b79058bf87a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:33.872175 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd71cd09-c455-4f40-b4d9-b79058bf87a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:33.873704 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd71cd09-c455-4f40-b4d9-b79058bf87a8 service neutron] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:33.887316 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1400/2772] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:40:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:40:34.324418 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179682fe-c481-4364-af33-f624b1963211 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:34.326724 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-179682fe-c481-4364-af33-f624b1963211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:34.340034 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179682fe-c481-4364-af33-f624b1963211 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:34.347940 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841824eb-8041-4ac2-93c5-05a1c789ae17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:34.348816 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841824eb-8041-4ac2-93c5-05a1c789ae17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:34.350235 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841824eb-8041-4ac2-93c5-05a1c789ae17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:34.350875 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841824eb-8041-4ac2-93c5-05a1c789ae17 admin admin] PATH_INFO: `/v3/projects/46e54d0ba39c4f2099007a429bc66881` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:34.352252 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841824eb-8041-4ac2-93c5-05a1c789ae17 admin admin] RBAC: Authorizing `identity:delete_project(project_id=46e54d0ba39c4f2099007a429bc66881)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:34.354759 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841824eb-8041-4ac2-93c5-05a1c789ae17 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:34.468422 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-841824eb-8041-4ac2-93c5-05a1c789ae17 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46e54d0ba39c4f2099007a429bc66881', 'request_id': 'req-841824eb-8041-4ac2-93c5-05a1c789ae17'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:34.546670 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-841824eb-8041-4ac2-93c5-05a1c789ae17 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46e54d0ba39c4f2099007a429bc66881', 'request_id': 'req-841824eb-8041-4ac2-93c5-05a1c789ae17'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:34.554252 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-841824eb-8041-4ac2-93c5-05a1c789ae17 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46e54d0ba39c4f2099007a429bc66881', 'request_id': 'req-841824eb-8041-4ac2-93c5-05a1c789ae17'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:34.568869 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-841824eb-8041-4ac2-93c5-05a1c789ae17 admin admin] The token cache is being invalidate because project 46e54d0ba39c4f2099007a429bc66881 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81732) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:40:34.569161 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-841824eb-8041-4ac2-93c5-05a1c789ae17 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:34.569823 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-841824eb-8041-4ac2-93c5-05a1c789ae17 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:34.571643 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1373/2773] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 24 17:40:34 2022] DELETE /identity/v3/projects/46e54d0ba39c4f2099007a429bc66881 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:40:34.576432 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eec7046-135a-488d-be5f-db847d44224f None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:34.576923 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eec7046-135a-488d-be5f-db847d44224f None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:34.577028 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eec7046-135a-488d-be5f-db847d44224f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:34.596468 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-0eec7046-135a-488d-be5f-db847d44224f None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:40:34.601294 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-0eec7046-135a-488d-be5f-db847d44224f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:34.615434 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:40:34.657718 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1401/2774] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:40:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:40:34.665721 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2620ad-51ad-4f9a-baac-62a79f30f5d7 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:34.666012 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2620ad-51ad-4f9a-baac-62a79f30f5d7 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:34.666290 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2620ad-51ad-4f9a-baac-62a79f30f5d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:34.687152 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-4b2620ad-51ad-4f9a-baac-62a79f30f5d7 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:40:34.688435 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-4b2620ad-51ad-4f9a-baac-62a79f30f5d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:34.697282 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1374/2775] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:40:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:40:34.701650 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eec7046-135a-488d-be5f-db847d44224f None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:34.712471 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eec7046-135a-488d-be5f-db847d44224f None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:34.717504 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b3c24e-e7d8-4e1c-beaf-52fecc320f16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:34.718202 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b3c24e-e7d8-4e1c-beaf-52fecc320f16 admin admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:34.721790 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b3c24e-e7d8-4e1c-beaf-52fecc320f16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:34.721790 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b3c24e-e7d8-4e1c-beaf-52fecc320f16 admin admin] PATH_INFO: `/v3/domains` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:34.721790 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b3c24e-e7d8-4e1c-beaf-52fecc320f16 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:40:34.721790 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b3c24e-e7d8-4e1c-beaf-52fecc320f16 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:34.721790 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b3c24e-e7d8-4e1c-beaf-52fecc320f16 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:34.740865 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1402/2776] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 24 17:40:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:40:34.745826 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2620ad-51ad-4f9a-baac-62a79f30f5d7 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:34.756093 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2620ad-51ad-4f9a-baac-62a79f30f5d7 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:34.760799 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f626f0ae-6659-4041-8e31-62c613797d24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:34.761407 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f626f0ae-6659-4041-8e31-62c613797d24 admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:34.761690 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f626f0ae-6659-4041-8e31-62c613797d24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:34.761965 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f626f0ae-6659-4041-8e31-62c613797d24 admin admin] PATH_INFO: `/v3/projects` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:34.762707 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f626f0ae-6659-4041-8e31-62c613797d24 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:34.764563 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f626f0ae-6659-4041-8e31-62c613797d24 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:34.776809 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:40:34.856276 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1375/2777] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 24 17:40:34 2022] POST /identity/v3/projects => generated 375 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:40:34.860740 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b3c24e-e7d8-4e1c-beaf-52fecc320f16 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:34.871138 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b3c24e-e7d8-4e1c-beaf-52fecc320f16 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:34.928951 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f089d48-40ca-471c-b248-8709bb3277ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:34.929595 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f089d48-40ca-471c-b248-8709bb3277ad admin admin] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:34.929871 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f089d48-40ca-471c-b248-8709bb3277ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:34.930140 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f089d48-40ca-471c-b248-8709bb3277ad admin admin] PATH_INFO: `/v3/users` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:34.930757 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f089d48-40ca-471c-b248-8709bb3277ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:34.932743 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f089d48-40ca-471c-b248-8709bb3277ad admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:35.013653 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1403/2778] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 24 17:40:34 2022] POST /identity/v3/users => generated 288 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:40:35.018695 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f626f0ae-6659-4041-8e31-62c613797d24 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:35.031219 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f626f0ae-6659-4041-8e31-62c613797d24 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:35.036459 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289bae5a-fd89-40dd-bad9-e932382d73c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:35.037148 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289bae5a-fd89-40dd-bad9-e932382d73c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:35.037424 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289bae5a-fd89-40dd-bad9-e932382d73c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:35.037694 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289bae5a-fd89-40dd-bad9-e932382d73c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:35.038360 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289bae5a-fd89-40dd-bad9-e932382d73c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:35.040161 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289bae5a-fd89-40dd-bad9-e932382d73c6 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:35.051310 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1376/2779] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:40:35 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:40:35.055473 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f089d48-40ca-471c-b248-8709bb3277ad admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:35.066415 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f089d48-40ca-471c-b248-8709bb3277ad admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:35.071085 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5253c4-4263-431d-aad6-199736babefd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:35.071712 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5253c4-4263-431d-aad6-199736babefd admin admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:35.071991 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5253c4-4263-431d-aad6-199736babefd admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:35.072262 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5253c4-4263-431d-aad6-199736babefd admin admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:35.072922 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5253c4-4263-431d-aad6-199736babefd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:35.074492 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5253c4-4263-431d-aad6-199736babefd admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:35.086829 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1404/2780] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:40:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:40:35.091067 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289bae5a-fd89-40dd-bad9-e932382d73c6 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:35.101407 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289bae5a-fd89-40dd-bad9-e932382d73c6 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:35.106214 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262430d9-1526-4c98-b79e-a93ddd4a6032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:35.106944 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262430d9-1526-4c98-b79e-a93ddd4a6032 admin admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:35.107221 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262430d9-1526-4c98-b79e-a93ddd4a6032 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:35.107496 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262430d9-1526-4c98-b79e-a93ddd4a6032 admin admin] PATH_INFO: `/v3/projects/f410101afcc246fa830d5661443fe628/users/34f0fc242a1c46ac8017b113df3f7e60/roles/60dc7e16aee746d5a3e8aa27d7d8525c` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:35.124113 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262430d9-1526-4c98-b79e-a93ddd4a6032 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f410101afcc246fa830d5661443fe628, user_id=34f0fc242a1c46ac8017b113df3f7e60, role_id=60dc7e16aee746d5a3e8aa27d7d8525c)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:35.127578 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262430d9-1526-4c98-b79e-a93ddd4a6032 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:35.220675 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1377/2781] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 24 17:40:35 2022] PUT /identity/v3/projects/f410101afcc246fa830d5661443fe628/users/34f0fc242a1c46ac8017b113df3f7e60/roles/60dc7e16aee746d5a3e8aa27d7d8525c => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:40:35.232356 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbbe6e8-02bb-4733-969e-c38336e15c74 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:35.232826 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbbe6e8-02bb-4733-969e-c38336e15c74 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:35.233129 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbbe6e8-02bb-4733-969e-c38336e15c74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:35.263740 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-ccbbe6e8-02bb-4733-969e-c38336e15c74 None None] MFA Rules not processed for user `34f0fc242a1c46ac8017b113df3f7e60`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:40:35.265598 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-ccbbe6e8-02bb-4733-969e-c38336e15c74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:35.343467 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1405/2782] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:40:35 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:40:35.357727 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262430d9-1526-4c98-b79e-a93ddd4a6032 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:35.359482 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-262430d9-1526-4c98-b79e-a93ddd4a6032 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:35.372987 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbbe6e8-02bb-4733-969e-c38336e15c74 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:35.375068 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-ccbbe6e8-02bb-4733-969e-c38336e15c74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:35.398257 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:40:35.441706 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262430d9-1526-4c98-b79e-a93ddd4a6032 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:35.449105 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbbe6e8-02bb-4733-969e-c38336e15c74 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:35.449747 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7003a825-e900-4a62-9a51-4868a78300ec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557aca9255cb4bbb8af52e796c5c5177', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:35.450787 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7003a825-e900-4a62-9a51-4868a78300ec service nova] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:35.451335 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7003a825-e900-4a62-9a51-4868a78300ec service nova] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:35.451602 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7003a825-e900-4a62-9a51-4868a78300ec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:35.457874 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9f27aa-47d2-402a-ba0a-92826268ea7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:35.458607 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9f27aa-47d2-402a-ba0a-92826268ea7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:35.458894 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9f27aa-47d2-402a-ba0a-92826268ea7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:35.459408 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9f27aa-47d2-402a-ba0a-92826268ea7f admin admin] PATH_INFO: `/v3/users/8d4898a6a5b2417d91d97eb33be44cf8` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:35.461120 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9f27aa-47d2-402a-ba0a-92826268ea7f admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d4898a6a5b2417d91d97eb33be44cf8)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:35.467298 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7003a825-e900-4a62-9a51-4868a78300ec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:35.467481 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9f27aa-47d2-402a-ba0a-92826268ea7f admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:35.467577 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7003a825-e900-4a62-9a51-4868a78300ec service nova] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:35.479640 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1378/2783] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:40:35 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:40:35.623850 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-8c9f27aa-47d2-402a-ba0a-92826268ea7f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d4898a6a5b2417d91d97eb33be44cf8'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:35.637894 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-8c9f27aa-47d2-402a-ba0a-92826268ea7f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d4898a6a5b2417d91d97eb33be44cf8'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:35.713289 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7003a825-e900-4a62-9a51-4868a78300ec service nova] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:35.721982 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-8c9f27aa-47d2-402a-ba0a-92826268ea7f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d4898a6a5b2417d91d97eb33be44cf8'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:35.722220 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7003a825-e900-4a62-9a51-4868a78300ec service nova] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:35.722681 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-8c9f27aa-47d2-402a-ba0a-92826268ea7f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d4898a6a5b2417d91d97eb33be44cf8'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:35.727194 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daedcd4-cebd-406b-a85b-98d22f9528d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:35.729578 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daedcd4-cebd-406b-a85b-98d22f9528d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:35.729578 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daedcd4-cebd-406b-a85b-98d22f9528d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:35.729578 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daedcd4-cebd-406b-a85b-98d22f9528d7 admin admin] PATH_INFO: `/v3/users/34f0fc242a1c46ac8017b113df3f7e60` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:35.729578 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7daedcd4-cebd-406b-a85b-98d22f9528d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=34f0fc242a1c46ac8017b113df3f7e60)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:35.730729 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-8c9f27aa-47d2-402a-ba0a-92826268ea7f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d4898a6a5b2417d91d97eb33be44cf8'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:35.731442 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7daedcd4-cebd-406b-a85b-98d22f9528d7 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:35.737485 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1406/2784] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 24 17:40:35 2022] DELETE /identity/v3/users/8d4898a6a5b2417d91d97eb33be44cf8 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:40:35.742531 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5305c01c-9b8a-4528-8ca3-b3dd9c5c5e97 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:35.742816 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5305c01c-9b8a-4528-8ca3-b3dd9c5c5e97 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:35.743115 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5305c01c-9b8a-4528-8ca3-b3dd9c5c5e97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:35.772365 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-5305c01c-9b8a-4528-8ca3-b3dd9c5c5e97 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:40:35.773816 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-5305c01c-9b8a-4528-8ca3-b3dd9c5c5e97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:35.834169 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1407/2785] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:40:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:40:35.846802 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5305c01c-9b8a-4528-8ca3-b3dd9c5c5e97 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:35.848247 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-5305c01c-9b8a-4528-8ca3-b3dd9c5c5e97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:35.891487 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:40:35.908342 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5305c01c-9b8a-4528-8ca3-b3dd9c5c5e97 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:35.912132 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55117da7-30c7-4873-b962-0745b2f89ce9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e9ed4d47aef4140bfb2d3bb798cc559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:35.912778 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55117da7-30c7-4873-b962-0745b2f89ce9 service neutron] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:35.913050 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55117da7-30c7-4873-b962-0745b2f89ce9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:35.913328 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55117da7-30c7-4873-b962-0745b2f89ce9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:35.915734 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-7daedcd4-cebd-406b-a85b-98d22f9528d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34f0fc242a1c46ac8017b113df3f7e60'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:35.922938 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55117da7-30c7-4873-b962-0745b2f89ce9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:35.924372 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55117da7-30c7-4873-b962-0745b2f89ce9 service neutron] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:35.934163 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1408/2786] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:40:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:40:36.008158 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-7daedcd4-cebd-406b-a85b-98d22f9528d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34f0fc242a1c46ac8017b113df3f7e60'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:36.008826 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-7daedcd4-cebd-406b-a85b-98d22f9528d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34f0fc242a1c46ac8017b113df3f7e60'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:36.017650 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-7daedcd4-cebd-406b-a85b-98d22f9528d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34f0fc242a1c46ac8017b113df3f7e60'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:36.018393 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-7daedcd4-cebd-406b-a85b-98d22f9528d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34f0fc242a1c46ac8017b113df3f7e60'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:36.037050 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1379/2787] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 24 17:40:35 2022] DELETE /identity/v3/users/34f0fc242a1c46ac8017b113df3f7e60 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:40:36.041865 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cde8b54-16b6-4f3c-b750-39066214cf2d None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:36.042151 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cde8b54-16b6-4f3c-b750-39066214cf2d None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:36.042772 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cde8b54-16b6-4f3c-b750-39066214cf2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:36.062807 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-2cde8b54-16b6-4f3c-b750-39066214cf2d None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:40:36.064050 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-2cde8b54-16b6-4f3c-b750-39066214cf2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:36.138358 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1409/2788] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:40:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:40:36.146624 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daedcd4-cebd-406b-a85b-98d22f9528d7 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:36.148864 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-7daedcd4-cebd-406b-a85b-98d22f9528d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:36.164424 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:40:36.211950 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daedcd4-cebd-406b-a85b-98d22f9528d7 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:36.215638 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82ed672-80f7-42f8-ab09-855e88297962 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e9ed4d47aef4140bfb2d3bb798cc559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:36.216257 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82ed672-80f7-42f8-ab09-855e88297962 service neutron] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:36.216531 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82ed672-80f7-42f8-ab09-855e88297962 service neutron] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:36.216798 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82ed672-80f7-42f8-ab09-855e88297962 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:36.227614 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82ed672-80f7-42f8-ab09-855e88297962 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:36.228594 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82ed672-80f7-42f8-ab09-855e88297962 service neutron] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:36.239689 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1380/2789] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:40:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:40:36.341100 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cde8b54-16b6-4f3c-b750-39066214cf2d None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:36.343568 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-2cde8b54-16b6-4f3c-b750-39066214cf2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:36.358953 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cde8b54-16b6-4f3c-b750-39066214cf2d None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:36.366020 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d14a1d3-0abe-495a-8adf-5a3382784a8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:36.366791 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d14a1d3-0abe-495a-8adf-5a3382784a8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:36.372145 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d14a1d3-0abe-495a-8adf-5a3382784a8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:36.372145 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d14a1d3-0abe-495a-8adf-5a3382784a8c admin admin] PATH_INFO: `/v3/projects/71b79a5db107412f97be56c247d337f1` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:36.372145 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d14a1d3-0abe-495a-8adf-5a3382784a8c admin admin] RBAC: Authorizing `identity:delete_project(project_id=71b79a5db107412f97be56c247d337f1)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:36.372145 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d14a1d3-0abe-495a-8adf-5a3382784a8c admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:36.499936 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-8d14a1d3-0abe-495a-8adf-5a3382784a8c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71b79a5db107412f97be56c247d337f1', 'request_id': 'req-8d14a1d3-0abe-495a-8adf-5a3382784a8c'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:36.574325 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-8d14a1d3-0abe-495a-8adf-5a3382784a8c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71b79a5db107412f97be56c247d337f1', 'request_id': 'req-8d14a1d3-0abe-495a-8adf-5a3382784a8c'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:36.582187 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-8d14a1d3-0abe-495a-8adf-5a3382784a8c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71b79a5db107412f97be56c247d337f1', 'request_id': 'req-8d14a1d3-0abe-495a-8adf-5a3382784a8c'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:36.594139 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-8d14a1d3-0abe-495a-8adf-5a3382784a8c admin admin] The token cache is being invalidate because project 71b79a5db107412f97be56c247d337f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81731) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:40:36.594549 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-8d14a1d3-0abe-495a-8adf-5a3382784a8c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:36.595294 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-8d14a1d3-0abe-495a-8adf-5a3382784a8c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:36.596903 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1410/2790] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 24 17:40:36 2022] DELETE /identity/v3/projects/71b79a5db107412f97be56c247d337f1 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:40:36.603508 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a3ec4d-c17a-4f10-86e6-dbaf6d3a141c None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:36.604040 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a3ec4d-c17a-4f10-86e6-dbaf6d3a141c None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:36.604314 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a3ec4d-c17a-4f10-86e6-dbaf6d3a141c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:36.624932 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-89a3ec4d-c17a-4f10-86e6-dbaf6d3a141c None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:40:36.626151 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-89a3ec4d-c17a-4f10-86e6-dbaf6d3a141c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:36.686295 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1381/2791] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:40:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:40:36.697154 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cd27a3-a886-4e0e-b29c-c64a20bad3a4 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:36.697690 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cd27a3-a886-4e0e-b29c-c64a20bad3a4 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:36.697819 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cd27a3-a886-4e0e-b29c-c64a20bad3a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:36.718499 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-47cd27a3-a886-4e0e-b29c-c64a20bad3a4 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:40:36.719742 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-47cd27a3-a886-4e0e-b29c-c64a20bad3a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:36.728352 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1411/2792] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:40:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:40:36.730520 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a3ec4d-c17a-4f10-86e6-dbaf6d3a141c None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:36.738627 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-89a3ec4d-c17a-4f10-86e6-dbaf6d3a141c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:36.738943 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cd27a3-a886-4e0e-b29c-c64a20bad3a4 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:36.745548 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a3ec4d-c17a-4f10-86e6-dbaf6d3a141c None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:36.748091 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cd27a3-a886-4e0e-b29c-c64a20bad3a4 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:36.751340 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29aca3e8-7097-43e7-8a95-9ec942d35460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:36.751989 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aca3e8-7097-43e7-8a95-9ec942d35460 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:36.752430 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aca3e8-7097-43e7-8a95-9ec942d35460 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:36.752765 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aca3e8-7097-43e7-8a95-9ec942d35460 admin admin] PATH_INFO: `/v3/projects/f410101afcc246fa830d5661443fe628` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:36.753925 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29aca3e8-7097-43e7-8a95-9ec942d35460 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f410101afcc246fa830d5661443fe628)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:36.755916 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29aca3e8-7097-43e7-8a95-9ec942d35460 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:36.759830 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bd8213-878f-47a3-83cb-e6491d0a7c97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:36.760927 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bd8213-878f-47a3-83cb-e6491d0a7c97 admin admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:36.761331 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bd8213-878f-47a3-83cb-e6491d0a7c97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:36.761613 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bd8213-878f-47a3-83cb-e6491d0a7c97 admin admin] PATH_INFO: `/v3/domains` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:36.762400 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bd8213-878f-47a3-83cb-e6491d0a7c97 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:40:36.762863 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bd8213-878f-47a3-83cb-e6491d0a7c97 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:36.767187 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bd8213-878f-47a3-83cb-e6491d0a7c97 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:36.778943 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1412/2793] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 24 17:40:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:40:36.784594 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bd8213-878f-47a3-83cb-e6491d0a7c97 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:36.794967 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bd8213-878f-47a3-83cb-e6491d0a7c97 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:36.800453 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290f4ba4-a7ca-4946-a00b-d939a3694a15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:36.801763 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290f4ba4-a7ca-4946-a00b-d939a3694a15 admin admin] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:36.801763 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290f4ba4-a7ca-4946-a00b-d939a3694a15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:36.801763 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290f4ba4-a7ca-4946-a00b-d939a3694a15 admin admin] PATH_INFO: `/v3/projects` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:36.802257 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290f4ba4-a7ca-4946-a00b-d939a3694a15 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:36.804171 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290f4ba4-a7ca-4946-a00b-d939a3694a15 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:36.867894 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-29aca3e8-7097-43e7-8a95-9ec942d35460 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f410101afcc246fa830d5661443fe628', 'request_id': 'req-29aca3e8-7097-43e7-8a95-9ec942d35460'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:36.942308 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1413/2794] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 24 17:40:36 2022] POST /identity/v3/projects => generated 383 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:40:36.946056 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-29aca3e8-7097-43e7-8a95-9ec942d35460 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f410101afcc246fa830d5661443fe628', 'request_id': 'req-29aca3e8-7097-43e7-8a95-9ec942d35460'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:36.946442 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290f4ba4-a7ca-4946-a00b-d939a3694a15 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:36.954141 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-29aca3e8-7097-43e7-8a95-9ec942d35460 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f410101afcc246fa830d5661443fe628', 'request_id': 'req-29aca3e8-7097-43e7-8a95-9ec942d35460'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:36.964724 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-29aca3e8-7097-43e7-8a95-9ec942d35460 admin admin] The token cache is being invalidate because project f410101afcc246fa830d5661443fe628 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81732) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:40:36.965017 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-29aca3e8-7097-43e7-8a95-9ec942d35460 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:36.965681 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-29aca3e8-7097-43e7-8a95-9ec942d35460 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:36.967253 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1382/2795] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 24 17:40:36 2022] DELETE /identity/v3/projects/f410101afcc246fa830d5661443fe628 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:40:36.972537 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473d17e3-6ef2-4a99-874f-9d7ed4ecda7d None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:36.972936 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473d17e3-6ef2-4a99-874f-9d7ed4ecda7d None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:36.973229 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473d17e3-6ef2-4a99-874f-9d7ed4ecda7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:36.982342 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:40:36.995310 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-473d17e3-6ef2-4a99-874f-9d7ed4ecda7d None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:40:36.996650 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-473d17e3-6ef2-4a99-874f-9d7ed4ecda7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:37.017395 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290f4ba4-a7ca-4946-a00b-d939a3694a15 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:37.022396 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32734c66-9f33-470e-9d1d-b7c538a1e220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:37.023038 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32734c66-9f33-470e-9d1d-b7c538a1e220 admin admin] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:37.023378 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32734c66-9f33-470e-9d1d-b7c538a1e220 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:37.023604 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32734c66-9f33-470e-9d1d-b7c538a1e220 admin admin] PATH_INFO: `/v3/users` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:37.024227 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32734c66-9f33-470e-9d1d-b7c538a1e220 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:37.026512 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32734c66-9f33-470e-9d1d-b7c538a1e220 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:37.033046 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:40:37.066304 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1383/2796] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:40:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:40:37.074160 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9294ff98-f0ff-4b7f-b2a4-107b39d3e36b None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:37.074427 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9294ff98-f0ff-4b7f-b2a4-107b39d3e36b None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:37.074704 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9294ff98-f0ff-4b7f-b2a4-107b39d3e36b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:37.094620 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-9294ff98-f0ff-4b7f-b2a4-107b39d3e36b None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:40:37.095844 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-9294ff98-f0ff-4b7f-b2a4-107b39d3e36b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:37.103700 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1384/2797] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:40:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:40:37.107604 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9294ff98-f0ff-4b7f-b2a4-107b39d3e36b None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:37.116952 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1414/2798] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 24 17:40:36 2022] POST /identity/v3/users => generated 292 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:40:37.117964 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9294ff98-f0ff-4b7f-b2a4-107b39d3e36b None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:37.121010 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32734c66-9f33-470e-9d1d-b7c538a1e220 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:37.122947 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-32734c66-9f33-470e-9d1d-b7c538a1e220 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:37.126006 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d85eab8-763a-4ad2-b0d4-74f0fe007d4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:37.126755 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d85eab8-763a-4ad2-b0d4-74f0fe007d4e admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:37.127069 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d85eab8-763a-4ad2-b0d4-74f0fe007d4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:37.127340 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d85eab8-763a-4ad2-b0d4-74f0fe007d4e admin admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:37.128036 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d85eab8-763a-4ad2-b0d4-74f0fe007d4e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:40:37.128310 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d85eab8-763a-4ad2-b0d4-74f0fe007d4e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:37.129999 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d85eab8-763a-4ad2-b0d4-74f0fe007d4e admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:37.136573 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32734c66-9f33-470e-9d1d-b7c538a1e220 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:37.141492 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b156d8-51c9-44a1-a8c9-fe581d481f85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:37.142051 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1385/2799] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 24 17:40:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:40:37.142199 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b156d8-51c9-44a1-a8c9-fe581d481f85 admin admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:37.142367 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b156d8-51c9-44a1-a8c9-fe581d481f85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:37.142685 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b156d8-51c9-44a1-a8c9-fe581d481f85 admin admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:37.143380 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b156d8-51c9-44a1-a8c9-fe581d481f85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:37.144993 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b156d8-51c9-44a1-a8c9-fe581d481f85 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:37.146597 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d85eab8-763a-4ad2-b0d4-74f0fe007d4e admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:37.155442 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1415/2800] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:40:37 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:40:37.157166 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d85eab8-763a-4ad2-b0d4-74f0fe007d4e admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:37.159649 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b156d8-51c9-44a1-a8c9-fe581d481f85 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:37.161941 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dda1a15-b062-4745-badb-59acb778e1d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:37.162539 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dda1a15-b062-4745-badb-59acb778e1d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:37.162815 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dda1a15-b062-4745-badb-59acb778e1d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:37.166106 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dda1a15-b062-4745-badb-59acb778e1d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:37.166106 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dda1a15-b062-4745-badb-59acb778e1d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:37.166106 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dda1a15-b062-4745-badb-59acb778e1d9 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:37.169281 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b156d8-51c9-44a1-a8c9-fe581d481f85 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:37.174815 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ead215-15fe-4ec2-b221-7e503ae95352 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:37.175449 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ead215-15fe-4ec2-b221-7e503ae95352 admin admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:37.175735 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ead215-15fe-4ec2-b221-7e503ae95352 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:37.176010 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ead215-15fe-4ec2-b221-7e503ae95352 admin admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:37.176682 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ead215-15fe-4ec2-b221-7e503ae95352 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:37.178308 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ead215-15fe-4ec2-b221-7e503ae95352 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:37.189771 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1416/2801] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:40:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:40:37.193920 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ead215-15fe-4ec2-b221-7e503ae95352 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:37.203507 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ead215-15fe-4ec2-b221-7e503ae95352 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:37.209197 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d605832-2be8-4a5c-967d-4df291bf5ae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:37.209893 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d605832-2be8-4a5c-967d-4df291bf5ae1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:37.210171 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d605832-2be8-4a5c-967d-4df291bf5ae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:37.210451 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d605832-2be8-4a5c-967d-4df291bf5ae1 admin admin] PATH_INFO: `/v3/projects/7ebcc5baa03b430bb50f4110c5a79df9/users/b4076a4c51a040a094aaae066772f9d2/roles/60dc7e16aee746d5a3e8aa27d7d8525c` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:37.226247 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d605832-2be8-4a5c-967d-4df291bf5ae1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ebcc5baa03b430bb50f4110c5a79df9, user_id=b4076a4c51a040a094aaae066772f9d2, role_id=60dc7e16aee746d5a3e8aa27d7d8525c)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:37.228893 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d605832-2be8-4a5c-967d-4df291bf5ae1 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:37.235128 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1386/2802] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 24 17:40:37 2022] POST /identity/v3/projects => generated 367 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:40:37.239289 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dda1a15-b062-4745-badb-59acb778e1d9 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:37.300085 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dda1a15-b062-4745-badb-59acb778e1d9 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:37.304967 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dc1ea9-24db-44cc-ad03-e4a62ec82f17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:37.305625 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dc1ea9-24db-44cc-ad03-e4a62ec82f17 admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:37.306279 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dc1ea9-24db-44cc-ad03-e4a62ec82f17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:37.306279 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dc1ea9-24db-44cc-ad03-e4a62ec82f17 admin admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:37.306749 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dc1ea9-24db-44cc-ad03-e4a62ec82f17 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:37.308683 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dc1ea9-24db-44cc-ad03-e4a62ec82f17 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:37.318009 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1417/2803] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 24 17:40:37 2022] PUT /identity/v3/projects/7ebcc5baa03b430bb50f4110c5a79df9/users/b4076a4c51a040a094aaae066772f9d2/roles/60dc7e16aee746d5a3e8aa27d7d8525c => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:40:37.322495 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0317c8-8310-4ee2-8d41-29d23eee9bbb None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:37.322792 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0317c8-8310-4ee2-8d41-29d23eee9bbb None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:37.323219 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0317c8-8310-4ee2-8d41-29d23eee9bbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:37.344003 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-4a0317c8-8310-4ee2-8d41-29d23eee9bbb None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:40:37.345208 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-4a0317c8-8310-4ee2-8d41-29d23eee9bbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:37.408024 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1418/2804] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:40:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:40:37.417850 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0317c8-8310-4ee2-8d41-29d23eee9bbb None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:37.419411 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-4a0317c8-8310-4ee2-8d41-29d23eee9bbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:37.419785 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1387/2805] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 24 17:40:37 2022] POST /identity/v3/users => generated 284 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:40:37.424137 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dc1ea9-24db-44cc-ad03-e4a62ec82f17 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:37.435856 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dc1ea9-24db-44cc-ad03-e4a62ec82f17 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:37.442173 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556ce280-1034-412e-b01e-da252c637ed4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:37.442950 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556ce280-1034-412e-b01e-da252c637ed4 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:37.443372 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556ce280-1034-412e-b01e-da252c637ed4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:37.443837 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556ce280-1034-412e-b01e-da252c637ed4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:37.444844 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556ce280-1034-412e-b01e-da252c637ed4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:37.446681 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556ce280-1034-412e-b01e-da252c637ed4 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:37.460819 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1388/2806] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:40:37 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:40:37.464767 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556ce280-1034-412e-b01e-da252c637ed4 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:37.474998 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556ce280-1034-412e-b01e-da252c637ed4 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:37.480195 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbf1a79-53a1-407e-97be-60d4e1da482b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:37.480813 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbf1a79-53a1-407e-97be-60d4e1da482b admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:37.481101 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbf1a79-53a1-407e-97be-60d4e1da482b admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:37.481383 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbf1a79-53a1-407e-97be-60d4e1da482b admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:37.482033 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbf1a79-53a1-407e-97be-60d4e1da482b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:37.484410 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbf1a79-53a1-407e-97be-60d4e1da482b admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:37.492810 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0317c8-8310-4ee2-8d41-29d23eee9bbb None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:37.496256 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1389/2807] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:40:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:40:37.496649 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692fbd2f-7b8b-4e85-b3e8-30a4744077c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e9ed4d47aef4140bfb2d3bb798cc559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:37.497331 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692fbd2f-7b8b-4e85-b3e8-30a4744077c7 service neutron] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:37.497611 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692fbd2f-7b8b-4e85-b3e8-30a4744077c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:37.497890 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692fbd2f-7b8b-4e85-b3e8-30a4744077c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:37.500869 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbf1a79-53a1-407e-97be-60d4e1da482b admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:37.511821 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbf1a79-53a1-407e-97be-60d4e1da482b admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:37.512565 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692fbd2f-7b8b-4e85-b3e8-30a4744077c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:37.514280 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692fbd2f-7b8b-4e85-b3e8-30a4744077c7 service neutron] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:37.518190 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830be0ff-6f5f-41d7-98af-672625c177c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:37.518873 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830be0ff-6f5f-41d7-98af-672625c177c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:37.519191 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830be0ff-6f5f-41d7-98af-672625c177c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:37.519466 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830be0ff-6f5f-41d7-98af-672625c177c8 admin admin] PATH_INFO: `/v3/projects/5cc3cab4d41a4efdb40e169dbb251d70/users/4b42ce8ef21f490db7ac465beca8a054/roles/60dc7e16aee746d5a3e8aa27d7d8525c` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:37.527000 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1419/2808] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:40:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:40:37.538356 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830be0ff-6f5f-41d7-98af-672625c177c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5cc3cab4d41a4efdb40e169dbb251d70, user_id=4b42ce8ef21f490db7ac465beca8a054, role_id=60dc7e16aee746d5a3e8aa27d7d8525c)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:37.541132 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830be0ff-6f5f-41d7-98af-672625c177c8 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:37.644371 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1390/2809] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 24 17:40:37 2022] PUT /identity/v3/projects/5cc3cab4d41a4efdb40e169dbb251d70/users/4b42ce8ef21f490db7ac465beca8a054/roles/60dc7e16aee746d5a3e8aa27d7d8525c => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:40:37.654724 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b299b2e-dd80-4112-94e2-7adae4a49a27 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:37.655045 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b299b2e-dd80-4112-94e2-7adae4a49a27 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:37.655324 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b299b2e-dd80-4112-94e2-7adae4a49a27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:37.675956 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:40:37.685169 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-6b299b2e-dd80-4112-94e2-7adae4a49a27 None None] MFA Rules not processed for user `4b42ce8ef21f490db7ac465beca8a054`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:40:37.686392 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-6b299b2e-dd80-4112-94e2-7adae4a49a27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:37.763655 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1420/2810] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:40:37 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:40:37.770840 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830be0ff-6f5f-41d7-98af-672625c177c8 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:37.819573 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:40:37.837011 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830be0ff-6f5f-41d7-98af-672625c177c8 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:37.841767 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf8e034-e5bd-4cb6-b2ad-49d642215583 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:37.842414 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf8e034-e5bd-4cb6-b2ad-49d642215583 admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:37.842694 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf8e034-e5bd-4cb6-b2ad-49d642215583 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:37.843042 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf8e034-e5bd-4cb6-b2ad-49d642215583 admin admin] PATH_INFO: `/v3/projects` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:37.843822 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf8e034-e5bd-4cb6-b2ad-49d642215583 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:37.845779 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf8e034-e5bd-4cb6-b2ad-49d642215583 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:37.943370 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1391/2811] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 24 17:40:37 2022] POST /identity/v3/projects => generated 365 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:40:37.947602 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b299b2e-dd80-4112-94e2-7adae4a49a27 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:37.959317 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b299b2e-dd80-4112-94e2-7adae4a49a27 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:38.037555 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24f380f-c4f9-475e-80d8-c40e9411c6f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:38.038195 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24f380f-c4f9-475e-80d8-c40e9411c6f7 admin admin] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:38.038471 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24f380f-c4f9-475e-80d8-c40e9411c6f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:38.038746 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24f380f-c4f9-475e-80d8-c40e9411c6f7 admin admin] PATH_INFO: `/v3/users` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:38.039428 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24f380f-c4f9-475e-80d8-c40e9411c6f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:38.041363 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24f380f-c4f9-475e-80d8-c40e9411c6f7 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:38.101223 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1421/2812] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 24 17:40:37 2022] POST /identity/v3/users => generated 289 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:40:38.105438 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf8e034-e5bd-4cb6-b2ad-49d642215583 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:38.116259 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf8e034-e5bd-4cb6-b2ad-49d642215583 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:38.121305 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b24333d-c72a-4683-ad48-d1784b07e979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:38.121935 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b24333d-c72a-4683-ad48-d1784b07e979 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:38.123278 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b24333d-c72a-4683-ad48-d1784b07e979 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:38.123278 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b24333d-c72a-4683-ad48-d1784b07e979 admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:38.123278 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b24333d-c72a-4683-ad48-d1784b07e979 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:38.124806 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b24333d-c72a-4683-ad48-d1784b07e979 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:38.136553 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1392/2813] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:40:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:40:38.141798 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24f380f-c4f9-475e-80d8-c40e9411c6f7 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:38.152297 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24f380f-c4f9-475e-80d8-c40e9411c6f7 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:38.157716 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e0078e-aa54-49a3-ab25-46b7119f0ea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:38.158363 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e0078e-aa54-49a3-ab25-46b7119f0ea9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:38.158640 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e0078e-aa54-49a3-ab25-46b7119f0ea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:38.158942 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e0078e-aa54-49a3-ab25-46b7119f0ea9 admin admin] PATH_INFO: `/v3/projects/649cf9981a27488bb407bf8111fd95fd/users/60df4136fd504843ad129750081eed6c/roles/c5346bc54abf445f9900edf94264c850` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:38.175015 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e0078e-aa54-49a3-ab25-46b7119f0ea9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=649cf9981a27488bb407bf8111fd95fd, user_id=60df4136fd504843ad129750081eed6c, role_id=c5346bc54abf445f9900edf94264c850)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:38.177653 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e0078e-aa54-49a3-ab25-46b7119f0ea9 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:38.255531 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1422/2814] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 24 17:40:38 2022] PUT /identity/v3/projects/649cf9981a27488bb407bf8111fd95fd/users/60df4136fd504843ad129750081eed6c/roles/c5346bc54abf445f9900edf94264c850 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:40:38.259957 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b24333d-c72a-4683-ad48-d1784b07e979 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:38.270442 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b24333d-c72a-4683-ad48-d1784b07e979 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:38.336684 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2490d1f-b83f-4ce3-9d0e-578b063680d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:38.337308 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2490d1f-b83f-4ce3-9d0e-578b063680d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:38.337642 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2490d1f-b83f-4ce3-9d0e-578b063680d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:38.338152 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2490d1f-b83f-4ce3-9d0e-578b063680d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:38.338805 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2490d1f-b83f-4ce3-9d0e-578b063680d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:38.340487 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2490d1f-b83f-4ce3-9d0e-578b063680d4 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:38.354766 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1393/2815] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:40:38 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:40:38.359301 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e0078e-aa54-49a3-ab25-46b7119f0ea9 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:38.375748 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e0078e-aa54-49a3-ab25-46b7119f0ea9 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:38.382670 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b098f18f-763c-4502-86c7-caa746cb5473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:38.383877 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b098f18f-763c-4502-86c7-caa746cb5473 admin admin] REQUEST_METHOD: `PUT` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:38.383877 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b098f18f-763c-4502-86c7-caa746cb5473 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:38.383877 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b098f18f-763c-4502-86c7-caa746cb5473 admin admin] PATH_INFO: `/v3/projects/649cf9981a27488bb407bf8111fd95fd/users/60df4136fd504843ad129750081eed6c/roles/c5346bc54abf445f9900edf94264c850` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:38.385819 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b098f18f-763c-4502-86c7-caa746cb5473 admin admin] RBAC: Authorizing `identity:create_grant(project_id=649cf9981a27488bb407bf8111fd95fd, user_id=60df4136fd504843ad129750081eed6c, role_id=c5346bc54abf445f9900edf94264c850)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:38.388577 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b098f18f-763c-4502-86c7-caa746cb5473 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:38.399846 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1423/2816] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 24 17:40:38 2022] PUT /identity/v3/projects/649cf9981a27488bb407bf8111fd95fd/users/60df4136fd504843ad129750081eed6c/roles/c5346bc54abf445f9900edf94264c850 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:40:38.411820 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6faee27-113f-4feb-91b1-e990bc7a4ffc None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:38.412109 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6faee27-113f-4feb-91b1-e990bc7a4ffc None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:38.412434 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6faee27-113f-4feb-91b1-e990bc7a4ffc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:38.442197 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-a6faee27-113f-4feb-91b1-e990bc7a4ffc None None] MFA Rules not processed for user `60df4136fd504843ad129750081eed6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:40:38.443939 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-a6faee27-113f-4feb-91b1-e990bc7a4ffc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:38.519190 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1394/2817] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:40:38 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:40:38.529865 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b098f18f-763c-4502-86c7-caa746cb5473 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:38.531840 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-b098f18f-763c-4502-86c7-caa746cb5473 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:38.599853 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b098f18f-763c-4502-86c7-caa746cb5473 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:38.603578 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92851723-2b36-4dcf-9621-c18685a7dc0a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be06246d47584b529150ceb98a5cfd95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:38.604309 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92851723-2b36-4dcf-9621-c18685a7dc0a service cinder] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:38.604594 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92851723-2b36-4dcf-9621-c18685a7dc0a service cinder] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:38.604950 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92851723-2b36-4dcf-9621-c18685a7dc0a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:38.611235 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:40:38.615959 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92851723-2b36-4dcf-9621-c18685a7dc0a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:38.617080 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92851723-2b36-4dcf-9621-c18685a7dc0a service cinder] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:38.627925 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1424/2818] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:40:38 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:40:38.841153 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6faee27-113f-4feb-91b1-e990bc7a4ffc None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:38.851755 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6faee27-113f-4feb-91b1-e990bc7a4ffc None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:38.855490 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdca3296-5147-400c-a5ad-5e3b5ca605bc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be06246d47584b529150ceb98a5cfd95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:38.856101 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdca3296-5147-400c-a5ad-5e3b5ca605bc service cinder] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:38.856385 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdca3296-5147-400c-a5ad-5e3b5ca605bc service cinder] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:38.856655 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdca3296-5147-400c-a5ad-5e3b5ca605bc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:38.891306 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:40:38.920083 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdca3296-5147-400c-a5ad-5e3b5ca605bc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:38.921889 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdca3296-5147-400c-a5ad-5e3b5ca605bc service cinder] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:38.933861 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1395/2819] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:40:38 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:40:38.982034 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34977950-0db9-4e9a-ac51-33d82dabf429 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:38.983141 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34977950-0db9-4e9a-ac51-33d82dabf429 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:38.983379 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34977950-0db9-4e9a-ac51-33d82dabf429 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:39.019878 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-34977950-0db9-4e9a-ac51-33d82dabf429 None None] MFA Rules not processed for user `b4076a4c51a040a094aaae066772f9d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:40:39.021159 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-34977950-0db9-4e9a-ac51-33d82dabf429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:39.105669 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1425/2820] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:40:38 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:40:39.117119 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdca3296-5147-400c-a5ad-5e3b5ca605bc service cinder] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:39.118532 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-cdca3296-5147-400c-a5ad-5e3b5ca605bc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:39.130118 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdca3296-5147-400c-a5ad-5e3b5ca605bc service cinder] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:39.134157 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf79303-4500-46e3-aeb1-26e04f53f6b5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be06246d47584b529150ceb98a5cfd95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:39.134770 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf79303-4500-46e3-aeb1-26e04f53f6b5 service cinder] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:39.135110 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf79303-4500-46e3-aeb1-26e04f53f6b5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:39.135399 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf79303-4500-46e3-aeb1-26e04f53f6b5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:39.145448 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf79303-4500-46e3-aeb1-26e04f53f6b5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:39.147012 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf79303-4500-46e3-aeb1-26e04f53f6b5 service cinder] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:39.156980 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1396/2821] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:40:39 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:40:41.993993 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96acf1df-4eb4-4924-b3e2-306d21786951 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:42.009286 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96acf1df-4eb4-4924-b3e2-306d21786951 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:42.009286 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96acf1df-4eb4-4924-b3e2-306d21786951 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:42.014328 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-96acf1df-4eb4-4924-b3e2-306d21786951 None None] MFA Rules not processed for user `b4076a4c51a040a094aaae066772f9d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:40:42.015608 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-96acf1df-4eb4-4924-b3e2-306d21786951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:42.025847 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1426/2822] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:40:41 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:40:42.282107 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf79303-4500-46e3-aeb1-26e04f53f6b5 service cinder] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:42.284020 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-caf79303-4500-46e3-aeb1-26e04f53f6b5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:42.378296 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf79303-4500-46e3-aeb1-26e04f53f6b5 service cinder] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:42.382393 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7802ba8-4c75-4d0d-b704-c89167161291 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e9ed4d47aef4140bfb2d3bb798cc559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:42.383737 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7802ba8-4c75-4d0d-b704-c89167161291 service neutron] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:42.384114 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7802ba8-4c75-4d0d-b704-c89167161291 service neutron] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:42.384618 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7802ba8-4c75-4d0d-b704-c89167161291 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:42.386214 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-b7802ba8-4c75-4d0d-b704-c89167161291 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:42.397857 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7802ba8-4c75-4d0d-b704-c89167161291 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:42.399429 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7802ba8-4c75-4d0d-b704-c89167161291 service neutron] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:42.411400 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1397/2823] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:40:42 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:40:46.914637 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96acf1df-4eb4-4924-b3e2-306d21786951 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:46.982307 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96acf1df-4eb4-4924-b3e2-306d21786951 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:46.986850 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcc477f-e8c9-4169-85bb-51710f4ad580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:46.987519 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcc477f-e8c9-4169-85bb-51710f4ad580 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:46.987800 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcc477f-e8c9-4169-85bb-51710f4ad580 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:46.988076 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcc477f-e8c9-4169-85bb-51710f4ad580 admin admin] PATH_INFO: `/v3/users/4b42ce8ef21f490db7ac465beca8a054` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:46.989274 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcc477f-e8c9-4169-85bb-51710f4ad580 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b42ce8ef21f490db7ac465beca8a054)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:46.991225 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcc477f-e8c9-4169-85bb-51710f4ad580 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:47.184133 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-4dcc477f-e8c9-4169-85bb-51710f4ad580 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b42ce8ef21f490db7ac465beca8a054'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:47.197205 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-4dcc477f-e8c9-4169-85bb-51710f4ad580 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b42ce8ef21f490db7ac465beca8a054'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:47.300485 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-4dcc477f-e8c9-4169-85bb-51710f4ad580 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b42ce8ef21f490db7ac465beca8a054'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:47.301376 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-4dcc477f-e8c9-4169-85bb-51710f4ad580 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b42ce8ef21f490db7ac465beca8a054'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:47.308552 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:40:47.311180 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-4dcc477f-e8c9-4169-85bb-51710f4ad580 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b42ce8ef21f490db7ac465beca8a054'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:47.323695 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1427/2824] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 24 17:40:46 2022] DELETE /identity/v3/users/4b42ce8ef21f490db7ac465beca8a054 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:40:47.328308 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c74f88-9e3b-40e7-b21c-a66f3c70acbb None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:47.328805 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c74f88-9e3b-40e7-b21c-a66f3c70acbb None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:47.328918 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c74f88-9e3b-40e7-b21c-a66f3c70acbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:47.355858 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-66c74f88-9e3b-40e7-b21c-a66f3c70acbb None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:40:47.357722 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-66c74f88-9e3b-40e7-b21c-a66f3c70acbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:47.433875 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1398/2825] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:40:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:40:47.447471 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcc477f-e8c9-4169-85bb-51710f4ad580 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:47.449140 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-4dcc477f-e8c9-4169-85bb-51710f4ad580 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:47.524236 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcc477f-e8c9-4169-85bb-51710f4ad580 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:47.530245 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87de648-27f1-44ac-8057-558340a2434a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e9ed4d47aef4140bfb2d3bb798cc559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:47.532082 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87de648-27f1-44ac-8057-558340a2434a service neutron] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:47.532548 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87de648-27f1-44ac-8057-558340a2434a service neutron] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:47.533069 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87de648-27f1-44ac-8057-558340a2434a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:47.548600 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87de648-27f1-44ac-8057-558340a2434a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:47.550726 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87de648-27f1-44ac-8057-558340a2434a service neutron] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:47.561357 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1428/2826] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:40:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:40:48.139230 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c74f88-9e3b-40e7-b21c-a66f3c70acbb None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:48.141290 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-66c74f88-9e3b-40e7-b21c-a66f3c70acbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:48.164996 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c74f88-9e3b-40e7-b21c-a66f3c70acbb None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:48.168751 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b42445-427d-4106-8e54-eddde13af0f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:48.169384 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b42445-427d-4106-8e54-eddde13af0f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:48.169667 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b42445-427d-4106-8e54-eddde13af0f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:48.169944 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b42445-427d-4106-8e54-eddde13af0f9 admin admin] PATH_INFO: `/v3/projects/5cc3cab4d41a4efdb40e169dbb251d70` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:48.171166 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b42445-427d-4106-8e54-eddde13af0f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5cc3cab4d41a4efdb40e169dbb251d70)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:48.174480 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b42445-427d-4106-8e54-eddde13af0f9 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:48.302069 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-e3b42445-427d-4106-8e54-eddde13af0f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5cc3cab4d41a4efdb40e169dbb251d70', 'request_id': 'req-e3b42445-427d-4106-8e54-eddde13af0f9'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:48.397382 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-e3b42445-427d-4106-8e54-eddde13af0f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5cc3cab4d41a4efdb40e169dbb251d70', 'request_id': 'req-e3b42445-427d-4106-8e54-eddde13af0f9'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:48.404574 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-e3b42445-427d-4106-8e54-eddde13af0f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5cc3cab4d41a4efdb40e169dbb251d70', 'request_id': 'req-e3b42445-427d-4106-8e54-eddde13af0f9'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:48.406672 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:40:48.416785 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-e3b42445-427d-4106-8e54-eddde13af0f9 admin admin] The token cache is being invalidate because project 5cc3cab4d41a4efdb40e169dbb251d70 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81732) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:40:48.417006 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-e3b42445-427d-4106-8e54-eddde13af0f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:48.417764 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-e3b42445-427d-4106-8e54-eddde13af0f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:48.419349 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1399/2827] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 24 17:40:48 2022] DELETE /identity/v3/projects/5cc3cab4d41a4efdb40e169dbb251d70 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:40:48.423773 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87de648-27f1-44ac-8057-558340a2434a service neutron] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:48.433634 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87de648-27f1-44ac-8057-558340a2434a service neutron] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:48.494806 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985f3d50-1d79-4449-8abe-9cb9e5693b68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:48.495503 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985f3d50-1d79-4449-8abe-9cb9e5693b68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:48.495777 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985f3d50-1d79-4449-8abe-9cb9e5693b68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:48.496106 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985f3d50-1d79-4449-8abe-9cb9e5693b68 admin admin] PATH_INFO: `/v3/users/60df4136fd504843ad129750081eed6c` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:48.497412 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985f3d50-1d79-4449-8abe-9cb9e5693b68 admin admin] RBAC: Authorizing `identity:delete_user(user_id=60df4136fd504843ad129750081eed6c)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:48.500152 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985f3d50-1d79-4449-8abe-9cb9e5693b68 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:48.818244 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-985f3d50-1d79-4449-8abe-9cb9e5693b68 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '60df4136fd504843ad129750081eed6c'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:48.830343 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-985f3d50-1d79-4449-8abe-9cb9e5693b68 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '60df4136fd504843ad129750081eed6c'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:48.933713 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-985f3d50-1d79-4449-8abe-9cb9e5693b68 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '60df4136fd504843ad129750081eed6c'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:48.934381 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-985f3d50-1d79-4449-8abe-9cb9e5693b68 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '60df4136fd504843ad129750081eed6c'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:48.942309 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-985f3d50-1d79-4449-8abe-9cb9e5693b68 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '60df4136fd504843ad129750081eed6c'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:48.948414 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1429/2828] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 24 17:40:48 2022] DELETE /identity/v3/users/60df4136fd504843ad129750081eed6c => generated 0 bytes in 525 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:40:49.399966 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b42445-427d-4106-8e54-eddde13af0f9 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:49.401494 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-e3b42445-427d-4106-8e54-eddde13af0f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:49.415943 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b42445-427d-4106-8e54-eddde13af0f9 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:49.477894 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f401d79-b56a-40d2-94a7-a3af1c5e2737 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:49.478523 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f401d79-b56a-40d2-94a7-a3af1c5e2737 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:49.478800 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f401d79-b56a-40d2-94a7-a3af1c5e2737 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:49.479099 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f401d79-b56a-40d2-94a7-a3af1c5e2737 admin admin] PATH_INFO: `/v3/projects/649cf9981a27488bb407bf8111fd95fd` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:49.480212 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f401d79-b56a-40d2-94a7-a3af1c5e2737 admin admin] RBAC: Authorizing `identity:delete_project(project_id=649cf9981a27488bb407bf8111fd95fd)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:49.488744 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f401d79-b56a-40d2-94a7-a3af1c5e2737 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:49.600047 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-9f401d79-b56a-40d2-94a7-a3af1c5e2737 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '649cf9981a27488bb407bf8111fd95fd', 'request_id': 'req-9f401d79-b56a-40d2-94a7-a3af1c5e2737'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:49.676503 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-9f401d79-b56a-40d2-94a7-a3af1c5e2737 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '649cf9981a27488bb407bf8111fd95fd', 'request_id': 'req-9f401d79-b56a-40d2-94a7-a3af1c5e2737'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:49.683768 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-9f401d79-b56a-40d2-94a7-a3af1c5e2737 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '649cf9981a27488bb407bf8111fd95fd', 'request_id': 'req-9f401d79-b56a-40d2-94a7-a3af1c5e2737'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:49.695076 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-9f401d79-b56a-40d2-94a7-a3af1c5e2737 admin admin] The token cache is being invalidate because project 649cf9981a27488bb407bf8111fd95fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81732) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:40:49.695349 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-9f401d79-b56a-40d2-94a7-a3af1c5e2737 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:49.696003 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-9f401d79-b56a-40d2-94a7-a3af1c5e2737 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:40:49.697541 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1400/2829] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 24 17:40:49 2022] DELETE /identity/v3/projects/649cf9981a27488bb407bf8111fd95fd => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:40:49.702540 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8ca5e1-db64-4b73-b90a-caf49266a064 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:49.703619 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8ca5e1-db64-4b73-b90a-caf49266a064 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:49.703619 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8ca5e1-db64-4b73-b90a-caf49266a064 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:49.724879 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-7c8ca5e1-db64-4b73-b90a-caf49266a064 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:40:49.726202 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-7c8ca5e1-db64-4b73-b90a-caf49266a064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:49.739262 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:40:49.789828 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1430/2830] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:40:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:40:49.798390 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c239437-5984-4dac-9d16-e1ea8de5ca17 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:49.798706 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c239437-5984-4dac-9d16-e1ea8de5ca17 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:49.799026 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c239437-5984-4dac-9d16-e1ea8de5ca17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:49.943637 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-9c239437-5984-4dac-9d16-e1ea8de5ca17 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:40:49.944732 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-9c239437-5984-4dac-9d16-e1ea8de5ca17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:49.953431 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1401/2831] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:40:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:40:49.958369 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8ca5e1-db64-4b73-b90a-caf49266a064 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:49.969894 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8ca5e1-db64-4b73-b90a-caf49266a064 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:49.975655 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d73129c-a4a9-46be-8d7e-9d2c3f667cd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:49.976291 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d73129c-a4a9-46be-8d7e-9d2c3f667cd3 admin admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:49.976555 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d73129c-a4a9-46be-8d7e-9d2c3f667cd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:49.976834 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d73129c-a4a9-46be-8d7e-9d2c3f667cd3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:49.977494 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d73129c-a4a9-46be-8d7e-9d2c3f667cd3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:40:49.977837 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d73129c-a4a9-46be-8d7e-9d2c3f667cd3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:49.979493 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d73129c-a4a9-46be-8d7e-9d2c3f667cd3 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:49.992533 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1431/2832] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 24 17:40:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:40:49.996738 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c239437-5984-4dac-9d16-e1ea8de5ca17 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:50.006880 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c239437-5984-4dac-9d16-e1ea8de5ca17 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:50.011679 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d1a81f-628b-4ead-a4b3-e793b8cb0093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:50.012260 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d1a81f-628b-4ead-a4b3-e793b8cb0093 admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:50.012532 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d1a81f-628b-4ead-a4b3-e793b8cb0093 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:50.012804 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d1a81f-628b-4ead-a4b3-e793b8cb0093 admin admin] PATH_INFO: `/v3/projects` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:50.013428 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d1a81f-628b-4ead-a4b3-e793b8cb0093 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:50.016004 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d1a81f-628b-4ead-a4b3-e793b8cb0093 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:50.103046 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1402/2833] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 24 17:40:49 2022] POST /identity/v3/projects => generated 377 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:40:50.107323 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d73129c-a4a9-46be-8d7e-9d2c3f667cd3 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:50.118281 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d73129c-a4a9-46be-8d7e-9d2c3f667cd3 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:50.177708 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca26cf1-4b2a-40b7-9488-b86e28979193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:50.178325 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca26cf1-4b2a-40b7-9488-b86e28979193 admin admin] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:50.178601 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca26cf1-4b2a-40b7-9488-b86e28979193 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:50.178880 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca26cf1-4b2a-40b7-9488-b86e28979193 admin admin] PATH_INFO: `/v3/users` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:50.179555 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca26cf1-4b2a-40b7-9488-b86e28979193 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:50.181455 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca26cf1-4b2a-40b7-9488-b86e28979193 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:50.294083 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1432/2834] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 24 17:40:50 2022] POST /identity/v3/users => generated 289 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:40:50.298231 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d1a81f-628b-4ead-a4b3-e793b8cb0093 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:50.308619 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d1a81f-628b-4ead-a4b3-e793b8cb0093 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:50.314009 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d971176-bc12-46e1-a2d5-01e2fb4a0d73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:50.318621 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d971176-bc12-46e1-a2d5-01e2fb4a0d73 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:50.318621 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d971176-bc12-46e1-a2d5-01e2fb4a0d73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:50.318621 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d971176-bc12-46e1-a2d5-01e2fb4a0d73 admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:50.318621 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d971176-bc12-46e1-a2d5-01e2fb4a0d73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:50.318621 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d971176-bc12-46e1-a2d5-01e2fb4a0d73 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:50.329066 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1403/2835] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:40:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:40:50.333415 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca26cf1-4b2a-40b7-9488-b86e28979193 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:50.343635 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca26cf1-4b2a-40b7-9488-b86e28979193 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:50.348272 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4114ad0-a10b-4328-ac56-5baa423e3d2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:50.348875 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4114ad0-a10b-4328-ac56-5baa423e3d2a admin admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:50.349160 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4114ad0-a10b-4328-ac56-5baa423e3d2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:50.349439 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4114ad0-a10b-4328-ac56-5baa423e3d2a admin admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:50.350079 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4114ad0-a10b-4328-ac56-5baa423e3d2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:50.351718 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4114ad0-a10b-4328-ac56-5baa423e3d2a admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:50.362406 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1433/2836] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:40:50 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:40:50.366582 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d971176-bc12-46e1-a2d5-01e2fb4a0d73 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:50.377441 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d971176-bc12-46e1-a2d5-01e2fb4a0d73 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:50.382121 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c28514-3b7e-4602-aae2-b448d027e911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:50.382764 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c28514-3b7e-4602-aae2-b448d027e911 admin admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:50.383072 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c28514-3b7e-4602-aae2-b448d027e911 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:50.383346 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c28514-3b7e-4602-aae2-b448d027e911 admin admin] PATH_INFO: `/v3/projects/bac2ff3bbbce45269dbf7b4828dce1a4/users/b21c20668b884aba81d5a7a54f583925/roles/60dc7e16aee746d5a3e8aa27d7d8525c` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:50.398408 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c28514-3b7e-4602-aae2-b448d027e911 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bac2ff3bbbce45269dbf7b4828dce1a4, user_id=b21c20668b884aba81d5a7a54f583925, role_id=60dc7e16aee746d5a3e8aa27d7d8525c)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:50.401061 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c28514-3b7e-4602-aae2-b448d027e911 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:50.460519 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1404/2837] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 24 17:40:50 2022] PUT /identity/v3/projects/bac2ff3bbbce45269dbf7b4828dce1a4/users/b21c20668b884aba81d5a7a54f583925/roles/60dc7e16aee746d5a3e8aa27d7d8525c => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:40:50.476139 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f47750-0391-4516-8acc-838b3042f9d5 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:50.476431 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f47750-0391-4516-8acc-838b3042f9d5 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:50.476733 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f47750-0391-4516-8acc-838b3042f9d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:50.506137 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-c5f47750-0391-4516-8acc-838b3042f9d5 None None] MFA Rules not processed for user `b21c20668b884aba81d5a7a54f583925`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:40:50.507386 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-c5f47750-0391-4516-8acc-838b3042f9d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:50.580555 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1434/2838] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:40:50 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:40:50.584997 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c28514-3b7e-4602-aae2-b448d027e911 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:50.602874 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:DELETE=9,keystone:SELECT=88,keystone:INSERT=3 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:40:50.646784 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c28514-3b7e-4602-aae2-b448d027e911 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:50.651426 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7387e918-0150-4034-88a5-2b4894bc0fd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:50.652031 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7387e918-0150-4034-88a5-2b4894bc0fd7 admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:50.652306 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7387e918-0150-4034-88a5-2b4894bc0fd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:50.652575 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7387e918-0150-4034-88a5-2b4894bc0fd7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:50.653207 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7387e918-0150-4034-88a5-2b4894bc0fd7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:50.655221 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7387e918-0150-4034-88a5-2b4894bc0fd7 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:50.752773 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1405/2839] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 24 17:40:50 2022] POST /identity/v3/projects => generated 377 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:40:50.756874 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f47750-0391-4516-8acc-838b3042f9d5 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:50.768446 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f47750-0391-4516-8acc-838b3042f9d5 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:50.777886 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:40:50.828229 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c85777f-800d-4a0a-9eb5-d133b2d22298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:50.828965 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c85777f-800d-4a0a-9eb5-d133b2d22298 admin admin] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:50.829317 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c85777f-800d-4a0a-9eb5-d133b2d22298 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:50.829658 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c85777f-800d-4a0a-9eb5-d133b2d22298 admin admin] PATH_INFO: `/v3/users` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:50.830718 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c85777f-800d-4a0a-9eb5-d133b2d22298 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:50.832564 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c85777f-800d-4a0a-9eb5-d133b2d22298 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:50.952101 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1435/2840] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 24 17:40:50 2022] POST /identity/v3/users => generated 289 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:40:50.956225 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7387e918-0150-4034-88a5-2b4894bc0fd7 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:50.966897 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7387e918-0150-4034-88a5-2b4894bc0fd7 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:50.972301 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd3beb4-ac5b-449b-bf0d-773aac9ebe40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:50.973186 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd3beb4-ac5b-449b-bf0d-773aac9ebe40 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:50.973527 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd3beb4-ac5b-449b-bf0d-773aac9ebe40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:50.973875 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd3beb4-ac5b-449b-bf0d-773aac9ebe40 admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:50.974587 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd3beb4-ac5b-449b-bf0d-773aac9ebe40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:50.976295 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd3beb4-ac5b-449b-bf0d-773aac9ebe40 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:50.988230 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1406/2841] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:40:50 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:40:50.992246 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c85777f-800d-4a0a-9eb5-d133b2d22298 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:51.002701 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c85777f-800d-4a0a-9eb5-d133b2d22298 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:51.007304 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3556ef5a-d0e8-481b-8712-838fb296fc18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:51.007894 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3556ef5a-d0e8-481b-8712-838fb296fc18 admin admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:51.008171 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3556ef5a-d0e8-481b-8712-838fb296fc18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:51.008442 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3556ef5a-d0e8-481b-8712-838fb296fc18 admin admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:51.009309 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3556ef5a-d0e8-481b-8712-838fb296fc18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:51.010866 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3556ef5a-d0e8-481b-8712-838fb296fc18 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:51.020843 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1436/2842] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:40:50 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:40:51.024967 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd3beb4-ac5b-449b-bf0d-773aac9ebe40 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:51.035952 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd3beb4-ac5b-449b-bf0d-773aac9ebe40 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:51.040517 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda86acc-20a7-4d39-bce5-dc8359436379 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:51.047504 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda86acc-20a7-4d39-bce5-dc8359436379 admin admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:51.047504 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda86acc-20a7-4d39-bce5-dc8359436379 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:51.047504 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda86acc-20a7-4d39-bce5-dc8359436379 admin admin] PATH_INFO: `/v3/projects/2174ae589466436d9327c157c456c828/users/4aad04e2812d430faf9ee76f3c331515/roles/60dc7e16aee746d5a3e8aa27d7d8525c` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:51.057474 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda86acc-20a7-4d39-bce5-dc8359436379 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2174ae589466436d9327c157c456c828, user_id=4aad04e2812d430faf9ee76f3c331515, role_id=60dc7e16aee746d5a3e8aa27d7d8525c)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:51.060119 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda86acc-20a7-4d39-bce5-dc8359436379 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:51.129858 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1407/2843] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 24 17:40:51 2022] PUT /identity/v3/projects/2174ae589466436d9327c157c456c828/users/4aad04e2812d430faf9ee76f3c331515/roles/60dc7e16aee746d5a3e8aa27d7d8525c => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:40:51.141968 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57239625-971e-4753-901f-400ef9c50379 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:51.142254 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57239625-971e-4753-901f-400ef9c50379 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:51.142525 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57239625-971e-4753-901f-400ef9c50379 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:51.171022 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-57239625-971e-4753-901f-400ef9c50379 None None] MFA Rules not processed for user `4aad04e2812d430faf9ee76f3c331515`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:40:51.172837 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-57239625-971e-4753-901f-400ef9c50379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:51.236617 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1437/2844] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:40:51 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:40:51.241023 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda86acc-20a7-4d39-bce5-dc8359436379 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:51.304339 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda86acc-20a7-4d39-bce5-dc8359436379 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:51.308965 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebce29ca-e847-4b6c-bb96-43cda6d46b8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:51.309569 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebce29ca-e847-4b6c-bb96-43cda6d46b8c admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:51.309847 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebce29ca-e847-4b6c-bb96-43cda6d46b8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:51.310215 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebce29ca-e847-4b6c-bb96-43cda6d46b8c admin admin] PATH_INFO: `/v3/projects` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:51.311006 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebce29ca-e847-4b6c-bb96-43cda6d46b8c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:51.312883 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebce29ca-e847-4b6c-bb96-43cda6d46b8c admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:51.405119 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1408/2845] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 24 17:40:51 2022] POST /identity/v3/projects => generated 375 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:40:51.409543 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57239625-971e-4753-901f-400ef9c50379 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:51.422446 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57239625-971e-4753-901f-400ef9c50379 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:51.480951 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2adedd2-723e-4ede-acd5-6656f81bf1a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:51.481571 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2adedd2-723e-4ede-acd5-6656f81bf1a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:51.481846 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2adedd2-723e-4ede-acd5-6656f81bf1a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:51.482122 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2adedd2-723e-4ede-acd5-6656f81bf1a9 admin admin] PATH_INFO: `/v3/users` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:51.482991 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2adedd2-723e-4ede-acd5-6656f81bf1a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:51.485206 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2adedd2-723e-4ede-acd5-6656f81bf1a9 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:51.581899 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1438/2846] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 24 17:40:51 2022] POST /identity/v3/users => generated 294 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:40:51.586308 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebce29ca-e847-4b6c-bb96-43cda6d46b8c admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:51.597483 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebce29ca-e847-4b6c-bb96-43cda6d46b8c admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:51.610694 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a36e8e-9c52-4e69-b345-efbd100577a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:51.614940 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a36e8e-9c52-4e69-b345-efbd100577a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:51.614940 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a36e8e-9c52-4e69-b345-efbd100577a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:51.614940 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a36e8e-9c52-4e69-b345-efbd100577a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:51.614940 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a36e8e-9c52-4e69-b345-efbd100577a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:51.614940 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a36e8e-9c52-4e69-b345-efbd100577a8 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:51.626066 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1409/2847] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:40:51 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:40:51.630323 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2adedd2-723e-4ede-acd5-6656f81bf1a9 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:51.640900 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2adedd2-723e-4ede-acd5-6656f81bf1a9 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:51.645551 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a4b449-771c-4935-9afb-ea5b1e83fad0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:51.646205 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a4b449-771c-4935-9afb-ea5b1e83fad0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:51.646480 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a4b449-771c-4935-9afb-ea5b1e83fad0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:51.646751 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a4b449-771c-4935-9afb-ea5b1e83fad0 admin admin] PATH_INFO: `/v3/projects/3d7739c62d9d4671a54cdb30ae81fa8a/users/ec5c09d364dc462aa4807ba1e7e0a064/roles/c5346bc54abf445f9900edf94264c850` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:51.655609 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:40:51.662638 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a4b449-771c-4935-9afb-ea5b1e83fad0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d7739c62d9d4671a54cdb30ae81fa8a, user_id=ec5c09d364dc462aa4807ba1e7e0a064, role_id=c5346bc54abf445f9900edf94264c850)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:51.665318 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a4b449-771c-4935-9afb-ea5b1e83fad0 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:51.753630 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1439/2848] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 24 17:40:51 2022] PUT /identity/v3/projects/3d7739c62d9d4671a54cdb30ae81fa8a/users/ec5c09d364dc462aa4807ba1e7e0a064/roles/c5346bc54abf445f9900edf94264c850 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:40:51.757750 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a36e8e-9c52-4e69-b345-efbd100577a8 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:51.769975 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a36e8e-9c52-4e69-b345-efbd100577a8 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:51.816173 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:40:51.838096 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bd5793-e806-4103-aa13-cb5db8f801d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:51.838966 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bd5793-e806-4103-aa13-cb5db8f801d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:51.838966 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bd5793-e806-4103-aa13-cb5db8f801d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:51.840134 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bd5793-e806-4103-aa13-cb5db8f801d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:51.840134 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bd5793-e806-4103-aa13-cb5db8f801d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:51.841570 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bd5793-e806-4103-aa13-cb5db8f801d0 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:51.855366 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1410/2849] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:40:51 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:40:51.860036 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a4b449-771c-4935-9afb-ea5b1e83fad0 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:51.871056 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a4b449-771c-4935-9afb-ea5b1e83fad0 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:51.875989 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8398e682-0504-46ff-8046-96977a9598f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:51.876653 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8398e682-0504-46ff-8046-96977a9598f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:51.876934 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8398e682-0504-46ff-8046-96977a9598f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:51.877211 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8398e682-0504-46ff-8046-96977a9598f2 admin admin] PATH_INFO: `/v3/projects/3d7739c62d9d4671a54cdb30ae81fa8a/users/ec5c09d364dc462aa4807ba1e7e0a064/roles/c5346bc54abf445f9900edf94264c850` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:51.879224 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8398e682-0504-46ff-8046-96977a9598f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d7739c62d9d4671a54cdb30ae81fa8a, user_id=ec5c09d364dc462aa4807ba1e7e0a064, role_id=c5346bc54abf445f9900edf94264c850)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:51.881834 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8398e682-0504-46ff-8046-96977a9598f2 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:51.893100 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1440/2850] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 24 17:40:51 2022] PUT /identity/v3/projects/3d7739c62d9d4671a54cdb30ae81fa8a/users/ec5c09d364dc462aa4807ba1e7e0a064/roles/c5346bc54abf445f9900edf94264c850 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:40:51.903355 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286696fc-36f9-46f4-88d6-1992baf0f2e7 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:51.907997 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286696fc-36f9-46f4-88d6-1992baf0f2e7 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:51.907997 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286696fc-36f9-46f4-88d6-1992baf0f2e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:51.931953 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-286696fc-36f9-46f4-88d6-1992baf0f2e7 None None] MFA Rules not processed for user `ec5c09d364dc462aa4807ba1e7e0a064`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:40:51.933269 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-286696fc-36f9-46f4-88d6-1992baf0f2e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:52.021746 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1411/2851] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:40:51 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:40:52.032776 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8398e682-0504-46ff-8046-96977a9598f2 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:52.034583 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-8398e682-0504-46ff-8046-96977a9598f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:40:52.100456 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8398e682-0504-46ff-8046-96977a9598f2 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:52.104036 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20bc2e1-8260-4a4e-a47b-0f225bc63e93 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be06246d47584b529150ceb98a5cfd95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:52.109632 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20bc2e1-8260-4a4e-a47b-0f225bc63e93 service cinder] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:52.109632 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20bc2e1-8260-4a4e-a47b-0f225bc63e93 service cinder] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:52.109632 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20bc2e1-8260-4a4e-a47b-0f225bc63e93 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:52.188500 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20bc2e1-8260-4a4e-a47b-0f225bc63e93 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:52.189980 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20bc2e1-8260-4a4e-a47b-0f225bc63e93 service cinder] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:52.200010 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1441/2852] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:40:52 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:40:55.212368 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286696fc-36f9-46f4-88d6-1992baf0f2e7 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:55.225639 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286696fc-36f9-46f4-88d6-1992baf0f2e7 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:55.229921 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02174bf5-fa14-4fb6-899d-95dec1318d74 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be06246d47584b529150ceb98a5cfd95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:55.230851 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02174bf5-fa14-4fb6-899d-95dec1318d74 service cinder] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:55.231319 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02174bf5-fa14-4fb6-899d-95dec1318d74 service cinder] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:55.231762 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02174bf5-fa14-4fb6-899d-95dec1318d74 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:55.300536 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02174bf5-fa14-4fb6-899d-95dec1318d74 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:55.302052 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02174bf5-fa14-4fb6-899d-95dec1318d74 service cinder] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:55.311528 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1412/2853] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:40:55 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:40:56.890984 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20bc2e1-8260-4a4e-a47b-0f225bc63e93 service cinder] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:40:56.900365 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20bc2e1-8260-4a4e-a47b-0f225bc63e93 service cinder] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:40:56.904075 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0ceb78-12b5-44bf-88de-c5cb06b039d9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be06246d47584b529150ceb98a5cfd95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:40:56.904678 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0ceb78-12b5-44bf-88de-c5cb06b039d9 service cinder] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:40:56.904959 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0ceb78-12b5-44bf-88de-c5cb06b039d9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:40:56.905234 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0ceb78-12b5-44bf-88de-c5cb06b039d9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:40:56.915035 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0ceb78-12b5-44bf-88de-c5cb06b039d9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:40:56.916454 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0ceb78-12b5-44bf-88de-c5cb06b039d9 service cinder] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:40:56.926384 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1442/2854] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:40:56 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:41:01.674948 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02174bf5-fa14-4fb6-899d-95dec1318d74 service cinder] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:01.749145 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02174bf5-fa14-4fb6-899d-95dec1318d74 service cinder] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:01.755422 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13168ab-9a81-4a8a-9353-66ebb7e9036b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:01.756068 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13168ab-9a81-4a8a-9353-66ebb7e9036b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:01.756350 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13168ab-9a81-4a8a-9353-66ebb7e9036b admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:01.756624 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13168ab-9a81-4a8a-9353-66ebb7e9036b admin admin] PATH_INFO: `/v3/users/b21c20668b884aba81d5a7a54f583925` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:01.757795 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13168ab-9a81-4a8a-9353-66ebb7e9036b admin admin] RBAC: Authorizing `identity:delete_user(user_id=b21c20668b884aba81d5a7a54f583925)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:01.759796 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13168ab-9a81-4a8a-9353-66ebb7e9036b admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:01.761472 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=100 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:02.033985 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-b13168ab-9a81-4a8a-9353-66ebb7e9036b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b21c20668b884aba81d5a7a54f583925'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:02.106022 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-b13168ab-9a81-4a8a-9353-66ebb7e9036b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b21c20668b884aba81d5a7a54f583925'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:02.106702 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-b13168ab-9a81-4a8a-9353-66ebb7e9036b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b21c20668b884aba81d5a7a54f583925'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:02.116306 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-b13168ab-9a81-4a8a-9353-66ebb7e9036b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b21c20668b884aba81d5a7a54f583925'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:02.117053 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-b13168ab-9a81-4a8a-9353-66ebb7e9036b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b21c20668b884aba81d5a7a54f583925'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:02.136230 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1413/2855] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 24 17:41:01 2022] DELETE /identity/v3/users/b21c20668b884aba81d5a7a54f583925 => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:41:02.140897 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9572a4ab-06ff-44b8-89b0-7546571cdb67 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:02.141383 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9572a4ab-06ff-44b8-89b0-7546571cdb67 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:02.141526 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9572a4ab-06ff-44b8-89b0-7546571cdb67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:02.162490 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-9572a4ab-06ff-44b8-89b0-7546571cdb67 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:41:02.163752 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-9572a4ab-06ff-44b8-89b0-7546571cdb67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:02.230589 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1443/2856] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:41:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:41:02.241374 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13168ab-9a81-4a8a-9353-66ebb7e9036b admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:02.242615 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-b13168ab-9a81-4a8a-9353-66ebb7e9036b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:02.305528 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13168ab-9a81-4a8a-9353-66ebb7e9036b admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:02.309706 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6916f0-7be7-4227-a70d-9b18d935ca13 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e9ed4d47aef4140bfb2d3bb798cc559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:02.310366 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6916f0-7be7-4227-a70d-9b18d935ca13 service neutron] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:02.310681 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6916f0-7be7-4227-a70d-9b18d935ca13 service neutron] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:02.310986 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6916f0-7be7-4227-a70d-9b18d935ca13 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:02.324031 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6916f0-7be7-4227-a70d-9b18d935ca13 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:02.325512 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6916f0-7be7-4227-a70d-9b18d935ca13 service neutron] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:02.336929 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1414/2857] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:41:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:41:02.815645 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9572a4ab-06ff-44b8-89b0-7546571cdb67 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:02.817144 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-9572a4ab-06ff-44b8-89b0-7546571cdb67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:02.834215 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9572a4ab-06ff-44b8-89b0-7546571cdb67 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:02.843062 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989a3a57-3142-456a-82ba-5effc64890b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:02.843934 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989a3a57-3142-456a-82ba-5effc64890b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:02.844310 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989a3a57-3142-456a-82ba-5effc64890b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:02.844691 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989a3a57-3142-456a-82ba-5effc64890b3 admin admin] PATH_INFO: `/v3/projects/bac2ff3bbbce45269dbf7b4828dce1a4` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:02.846048 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989a3a57-3142-456a-82ba-5effc64890b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bac2ff3bbbce45269dbf7b4828dce1a4)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:02.848494 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989a3a57-3142-456a-82ba-5effc64890b3 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:02.861623 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:02.994979 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-989a3a57-3142-456a-82ba-5effc64890b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bac2ff3bbbce45269dbf7b4828dce1a4', 'request_id': 'req-989a3a57-3142-456a-82ba-5effc64890b3'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:03.081636 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-989a3a57-3142-456a-82ba-5effc64890b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bac2ff3bbbce45269dbf7b4828dce1a4', 'request_id': 'req-989a3a57-3142-456a-82ba-5effc64890b3'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:03.088649 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-989a3a57-3142-456a-82ba-5effc64890b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bac2ff3bbbce45269dbf7b4828dce1a4', 'request_id': 'req-989a3a57-3142-456a-82ba-5effc64890b3'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:03.100568 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-989a3a57-3142-456a-82ba-5effc64890b3 admin admin] The token cache is being invalidate because project bac2ff3bbbce45269dbf7b4828dce1a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81731) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:41:03.100845 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-989a3a57-3142-456a-82ba-5effc64890b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:03.101519 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-989a3a57-3142-456a-82ba-5effc64890b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:03.103168 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1444/2858] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 24 17:41:02 2022] DELETE /identity/v3/projects/bac2ff3bbbce45269dbf7b4828dce1a4 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:41:03.107242 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6916f0-7be7-4227-a70d-9b18d935ca13 service neutron] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:03.117837 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6916f0-7be7-4227-a70d-9b18d935ca13 service neutron] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:03.182431 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbdec06-041a-43dd-abab-569466276078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:03.183206 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbdec06-041a-43dd-abab-569466276078 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:03.183490 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbdec06-041a-43dd-abab-569466276078 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:03.183775 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbdec06-041a-43dd-abab-569466276078 admin admin] PATH_INFO: `/v3/users/4aad04e2812d430faf9ee76f3c331515` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:03.184941 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbdec06-041a-43dd-abab-569466276078 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4aad04e2812d430faf9ee76f3c331515)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:03.186854 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbdec06-041a-43dd-abab-569466276078 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:03.215139 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:03.387217 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-6fbdec06-041a-43dd-abab-569466276078 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4aad04e2812d430faf9ee76f3c331515'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:03.465889 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-6fbdec06-041a-43dd-abab-569466276078 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4aad04e2812d430faf9ee76f3c331515'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:03.466539 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-6fbdec06-041a-43dd-abab-569466276078 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4aad04e2812d430faf9ee76f3c331515'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:03.478060 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-6fbdec06-041a-43dd-abab-569466276078 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4aad04e2812d430faf9ee76f3c331515'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:03.478715 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-6fbdec06-041a-43dd-abab-569466276078 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4aad04e2812d430faf9ee76f3c331515'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:03.499087 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1415/2859] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 24 17:41:03 2022] DELETE /identity/v3/users/4aad04e2812d430faf9ee76f3c331515 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:41:04.024140 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989a3a57-3142-456a-82ba-5effc64890b3 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:04.025658 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-989a3a57-3142-456a-82ba-5effc64890b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:04.044380 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989a3a57-3142-456a-82ba-5effc64890b3 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:04.122402 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6225aff6-843b-4e1b-b289-0c56e85efb04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:04.123445 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6225aff6-843b-4e1b-b289-0c56e85efb04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:04.123961 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6225aff6-843b-4e1b-b289-0c56e85efb04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:04.124329 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6225aff6-843b-4e1b-b289-0c56e85efb04 admin admin] PATH_INFO: `/v3/projects/2174ae589466436d9327c157c456c828` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:04.125578 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6225aff6-843b-4e1b-b289-0c56e85efb04 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2174ae589466436d9327c157c456c828)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:04.128442 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6225aff6-843b-4e1b-b289-0c56e85efb04 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:04.267987 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-6225aff6-843b-4e1b-b289-0c56e85efb04 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2174ae589466436d9327c157c456c828', 'request_id': 'req-6225aff6-843b-4e1b-b289-0c56e85efb04'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:04.323702 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-6225aff6-843b-4e1b-b289-0c56e85efb04 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2174ae589466436d9327c157c456c828', 'request_id': 'req-6225aff6-843b-4e1b-b289-0c56e85efb04'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:04.334177 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-6225aff6-843b-4e1b-b289-0c56e85efb04 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2174ae589466436d9327c157c456c828', 'request_id': 'req-6225aff6-843b-4e1b-b289-0c56e85efb04'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:04.347955 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-6225aff6-843b-4e1b-b289-0c56e85efb04 admin admin] The token cache is being invalidate because project 2174ae589466436d9327c157c456c828 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81731) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:41:04.348291 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-6225aff6-843b-4e1b-b289-0c56e85efb04 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:04.348974 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-6225aff6-843b-4e1b-b289-0c56e85efb04 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:04.350520 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1445/2860] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 24 17:41:04 2022] DELETE /identity/v3/projects/2174ae589466436d9327c157c456c828 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:41:04.354841 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbdec06-041a-43dd-abab-569466276078 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:04.356893 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-6fbdec06-041a-43dd-abab-569466276078 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:04.377506 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbdec06-041a-43dd-abab-569466276078 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:04.456904 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd93ef0-153e-4c61-852d-94828a6e13be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:04.457582 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd93ef0-153e-4c61-852d-94828a6e13be admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:04.457855 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd93ef0-153e-4c61-852d-94828a6e13be admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:04.458128 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd93ef0-153e-4c61-852d-94828a6e13be admin admin] PATH_INFO: `/v3/users/ec5c09d364dc462aa4807ba1e7e0a064` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:04.459381 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd93ef0-153e-4c61-852d-94828a6e13be admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec5c09d364dc462aa4807ba1e7e0a064)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:04.461275 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd93ef0-153e-4c61-852d-94828a6e13be admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:04.662691 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-5cd93ef0-153e-4c61-852d-94828a6e13be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec5c09d364dc462aa4807ba1e7e0a064'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:04.725198 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-5cd93ef0-153e-4c61-852d-94828a6e13be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec5c09d364dc462aa4807ba1e7e0a064'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:04.726145 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-5cd93ef0-153e-4c61-852d-94828a6e13be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec5c09d364dc462aa4807ba1e7e0a064'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:04.736079 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-5cd93ef0-153e-4c61-852d-94828a6e13be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec5c09d364dc462aa4807ba1e7e0a064'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:04.736839 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-5cd93ef0-153e-4c61-852d-94828a6e13be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec5c09d364dc462aa4807ba1e7e0a064'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:04.756004 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1416/2861] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 24 17:41:04 2022] DELETE /identity/v3/users/ec5c09d364dc462aa4807ba1e7e0a064 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:41:05.302291 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6225aff6-843b-4e1b-b289-0c56e85efb04 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:05.303801 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-6225aff6-843b-4e1b-b289-0c56e85efb04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:05.319256 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6225aff6-843b-4e1b-b289-0c56e85efb04 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:05.354188 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:05.388771 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb2e119-a7f9-4c30-bbee-c83258c998f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:05.389412 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb2e119-a7f9-4c30-bbee-c83258c998f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:05.389696 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb2e119-a7f9-4c30-bbee-c83258c998f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:05.389978 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb2e119-a7f9-4c30-bbee-c83258c998f7 admin admin] PATH_INFO: `/v3/projects/3d7739c62d9d4671a54cdb30ae81fa8a` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:05.391143 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb2e119-a7f9-4c30-bbee-c83258c998f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d7739c62d9d4671a54cdb30ae81fa8a)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:05.393682 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb2e119-a7f9-4c30-bbee-c83258c998f7 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:05.534628 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-ceb2e119-a7f9-4c30-bbee-c83258c998f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d7739c62d9d4671a54cdb30ae81fa8a', 'request_id': 'req-ceb2e119-a7f9-4c30-bbee-c83258c998f7'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:05.615235 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-ceb2e119-a7f9-4c30-bbee-c83258c998f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d7739c62d9d4671a54cdb30ae81fa8a', 'request_id': 'req-ceb2e119-a7f9-4c30-bbee-c83258c998f7'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:05.624837 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-ceb2e119-a7f9-4c30-bbee-c83258c998f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d7739c62d9d4671a54cdb30ae81fa8a', 'request_id': 'req-ceb2e119-a7f9-4c30-bbee-c83258c998f7'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:05.636798 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-ceb2e119-a7f9-4c30-bbee-c83258c998f7 admin admin] The token cache is being invalidate because project 3d7739c62d9d4671a54cdb30ae81fa8a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81731) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:41:05.637088 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-ceb2e119-a7f9-4c30-bbee-c83258c998f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:05.637786 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-ceb2e119-a7f9-4c30-bbee-c83258c998f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:41:05.639308 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1446/2862] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 24 17:41:05 2022] DELETE /identity/v3/projects/3d7739c62d9d4671a54cdb30ae81fa8a => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:41:05.644769 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d903f208-b058-4131-b0b4-0b3137e20008 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:05.645361 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d903f208-b058-4131-b0b4-0b3137e20008 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:05.645361 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d903f208-b058-4131-b0b4-0b3137e20008 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:05.666471 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-d903f208-b058-4131-b0b4-0b3137e20008 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:41:05.667714 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-d903f208-b058-4131-b0b4-0b3137e20008 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:05.691424 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:05.747947 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1417/2863] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:41:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 17:41:05.759424 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcad403-968b-4f8c-a87d-8e62b426d737 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:05.759764 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcad403-968b-4f8c-a87d-8e62b426d737 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:05.759974 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcad403-968b-4f8c-a87d-8e62b426d737 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:05.780418 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-cbcad403-968b-4f8c-a87d-8e62b426d737 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:41:05.782077 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-cbcad403-968b-4f8c-a87d-8e62b426d737 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:05.794773 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1447/2864] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:41:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:41:05.799908 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d903f208-b058-4131-b0b4-0b3137e20008 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:05.810278 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d903f208-b058-4131-b0b4-0b3137e20008 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:05.815047 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4918d5da-4d0f-4b0d-8af5-e81b03347c98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:05.815948 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4918d5da-4d0f-4b0d-8af5-e81b03347c98 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:05.816403 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4918d5da-4d0f-4b0d-8af5-e81b03347c98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:05.816846 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4918d5da-4d0f-4b0d-8af5-e81b03347c98 admin admin] PATH_INFO: `/v3/domains` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:05.818000 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4918d5da-4d0f-4b0d-8af5-e81b03347c98 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81732) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:41:05.818569 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4918d5da-4d0f-4b0d-8af5-e81b03347c98 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:05.820759 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4918d5da-4d0f-4b0d-8af5-e81b03347c98 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:05.833870 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1418/2865] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 24 17:41:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:41:05.838393 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcad403-968b-4f8c-a87d-8e62b426d737 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:05.848325 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcad403-968b-4f8c-a87d-8e62b426d737 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:05.854169 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b177760-ed56-493a-8f4e-6d3d6760ed76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:05.854979 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b177760-ed56-493a-8f4e-6d3d6760ed76 admin admin] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:05.854979 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b177760-ed56-493a-8f4e-6d3d6760ed76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:05.856090 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b177760-ed56-493a-8f4e-6d3d6760ed76 admin admin] PATH_INFO: `/v3/projects` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:05.856090 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b177760-ed56-493a-8f4e-6d3d6760ed76 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:05.857950 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b177760-ed56-493a-8f4e-6d3d6760ed76 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:05.941570 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1448/2866] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 24 17:41:05 2022] POST /identity/v3/projects => generated 371 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:41:05.947191 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4918d5da-4d0f-4b0d-8af5-e81b03347c98 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:05.958820 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4918d5da-4d0f-4b0d-8af5-e81b03347c98 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:06.023102 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973bf600-0d9c-4edb-8d0a-742c9d622df4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:06.023721 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973bf600-0d9c-4edb-8d0a-742c9d622df4 admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:06.024006 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973bf600-0d9c-4edb-8d0a-742c9d622df4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:06.024281 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973bf600-0d9c-4edb-8d0a-742c9d622df4 admin admin] PATH_INFO: `/v3/users` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:06.024907 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973bf600-0d9c-4edb-8d0a-742c9d622df4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:06.026823 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973bf600-0d9c-4edb-8d0a-742c9d622df4 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:06.111762 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1419/2867] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 24 17:41:05 2022] POST /identity/v3/users => generated 286 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:41:06.115482 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b177760-ed56-493a-8f4e-6d3d6760ed76 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:06.126572 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b177760-ed56-493a-8f4e-6d3d6760ed76 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:06.131499 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be69dd40-109d-42e3-b0c0-5d844e0b1252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:06.132105 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be69dd40-109d-42e3-b0c0-5d844e0b1252 admin admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:06.132400 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be69dd40-109d-42e3-b0c0-5d844e0b1252 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:06.139637 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be69dd40-109d-42e3-b0c0-5d844e0b1252 admin admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:06.139637 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be69dd40-109d-42e3-b0c0-5d844e0b1252 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:06.139637 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be69dd40-109d-42e3-b0c0-5d844e0b1252 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:06.149765 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1449/2868] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:41:06 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:41:06.154114 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973bf600-0d9c-4edb-8d0a-742c9d622df4 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:06.165686 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973bf600-0d9c-4edb-8d0a-742c9d622df4 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:06.171604 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25799ed9-ea00-4b9f-956b-b71b80e27209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:06.172259 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25799ed9-ea00-4b9f-956b-b71b80e27209 admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:06.172888 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25799ed9-ea00-4b9f-956b-b71b80e27209 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:06.173355 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25799ed9-ea00-4b9f-956b-b71b80e27209 admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:06.174446 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25799ed9-ea00-4b9f-956b-b71b80e27209 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:06.177877 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25799ed9-ea00-4b9f-956b-b71b80e27209 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:06.191661 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1420/2869] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:41:06 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:41:06.197142 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be69dd40-109d-42e3-b0c0-5d844e0b1252 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:06.208387 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be69dd40-109d-42e3-b0c0-5d844e0b1252 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:06.213534 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fa310a-afe0-458d-8069-9391495d1300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:06.214277 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fa310a-afe0-458d-8069-9391495d1300 admin admin] REQUEST_METHOD: `PUT` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:06.214564 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fa310a-afe0-458d-8069-9391495d1300 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:06.214870 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fa310a-afe0-458d-8069-9391495d1300 admin admin] PATH_INFO: `/v3/projects/20e6abbbc746457d87ff2ba46899afdc/users/199eb0649d2741fa8158af1736a14311/roles/60dc7e16aee746d5a3e8aa27d7d8525c` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:06.232123 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fa310a-afe0-458d-8069-9391495d1300 admin admin] RBAC: Authorizing `identity:create_grant(project_id=20e6abbbc746457d87ff2ba46899afdc, user_id=199eb0649d2741fa8158af1736a14311, role_id=60dc7e16aee746d5a3e8aa27d7d8525c)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:06.235015 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fa310a-afe0-458d-8069-9391495d1300 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:06.370573 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1450/2870] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 24 17:41:06 2022] PUT /identity/v3/projects/20e6abbbc746457d87ff2ba46899afdc/users/199eb0649d2741fa8158af1736a14311/roles/60dc7e16aee746d5a3e8aa27d7d8525c => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:41:06.384106 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93829588-5d2e-46fa-a2e5-64c920b667e6 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:06.384468 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93829588-5d2e-46fa-a2e5-64c920b667e6 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:06.384639 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93829588-5d2e-46fa-a2e5-64c920b667e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:06.416210 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-93829588-5d2e-46fa-a2e5-64c920b667e6 None None] MFA Rules not processed for user `199eb0649d2741fa8158af1736a14311`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:41:06.417438 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-93829588-5d2e-46fa-a2e5-64c920b667e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:06.489675 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1421/2871] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:41:06 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:41:06.502931 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fa310a-afe0-458d-8069-9391495d1300 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:06.505457 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-e9fa310a-afe0-458d-8069-9391495d1300 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:06.582501 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fa310a-afe0-458d-8069-9391495d1300 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:06.586781 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521d327b-6361-4ab9-893e-cdd65c7877d1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3553e522363046f6828fd61b2a0642ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:06.587441 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521d327b-6361-4ab9-893e-cdd65c7877d1 service glance] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:06.587724 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521d327b-6361-4ab9-893e-cdd65c7877d1 service glance] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:06.588007 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521d327b-6361-4ab9-893e-cdd65c7877d1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:06.601253 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521d327b-6361-4ab9-893e-cdd65c7877d1 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:06.602714 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521d327b-6361-4ab9-893e-cdd65c7877d1 service glance] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:06.607031 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:06.614557 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1451/2872] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:41:06 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:41:16.486694 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=90,keystone:INSERT=3 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:16.739039 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=1 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:41:24.234076 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93829588-5d2e-46fa-a2e5-64c920b667e6 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:41:24.236179 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-93829588-5d2e-46fa-a2e5-64c920b667e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:24.302885 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93829588-5d2e-46fa-a2e5-64c920b667e6 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:41:24.302885 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5126f286-f0eb-4e01-a2ea-77ac1fc8557e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be06246d47584b529150ceb98a5cfd95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:41:24.302885 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5126f286-f0eb-4e01-a2ea-77ac1fc8557e service cinder] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:41:24.302885 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5126f286-f0eb-4e01-a2ea-77ac1fc8557e service cinder] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:41:24.303806 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5126f286-f0eb-4e01-a2ea-77ac1fc8557e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:41:24.303806 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-5126f286-f0eb-4e01-a2ea-77ac1fc8557e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:41:24.375296 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5126f286-f0eb-4e01-a2ea-77ac1fc8557e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:41:24.377399 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5126f286-f0eb-4e01-a2ea-77ac1fc8557e service cinder] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:41:24.388814 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1422/2873] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:41:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:41:34.383693 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=54 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:42:00.629353 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521d327b-6361-4ab9-893e-cdd65c7877d1 service glance] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:00.638773 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521d327b-6361-4ab9-893e-cdd65c7877d1 service glance] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:00.642976 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63532cf-e4e4-4c5b-950a-c4da4fdf20cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be06246d47584b529150ceb98a5cfd95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:00.643580 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63532cf-e4e4-4c5b-950a-c4da4fdf20cd service cinder] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:00.644241 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63532cf-e4e4-4c5b-950a-c4da4fdf20cd service cinder] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:00.644241 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63532cf-e4e4-4c5b-950a-c4da4fdf20cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:00.646972 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-e63532cf-e4e4-4c5b-950a-c4da4fdf20cd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:00.661255 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63532cf-e4e4-4c5b-950a-c4da4fdf20cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:00.662682 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63532cf-e4e4-4c5b-950a-c4da4fdf20cd service cinder] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:00.674249 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1452/2874] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:42:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:42:02.719912 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5126f286-f0eb-4e01-a2ea-77ac1fc8557e service cinder] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:02.783267 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5126f286-f0eb-4e01-a2ea-77ac1fc8557e service cinder] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:02.788020 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bfc822-8de9-4796-a311-3e836b54637c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:02.788883 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bfc822-8de9-4796-a311-3e836b54637c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:02.789062 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bfc822-8de9-4796-a311-3e836b54637c admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:02.789333 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bfc822-8de9-4796-a311-3e836b54637c admin admin] PATH_INFO: `/v3/users/199eb0649d2741fa8158af1736a14311` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:02.790451 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bfc822-8de9-4796-a311-3e836b54637c admin admin] RBAC: Authorizing `identity:delete_user(user_id=199eb0649d2741fa8158af1736a14311)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:02.792397 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bfc822-8de9-4796-a311-3e836b54637c admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:03.001736 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-89bfc822-8de9-4796-a311-3e836b54637c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '199eb0649d2741fa8158af1736a14311'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:03.111357 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-89bfc822-8de9-4796-a311-3e836b54637c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '199eb0649d2741fa8158af1736a14311'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:03.111993 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-89bfc822-8de9-4796-a311-3e836b54637c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '199eb0649d2741fa8158af1736a14311'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:03.122827 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-89bfc822-8de9-4796-a311-3e836b54637c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '199eb0649d2741fa8158af1736a14311'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:03.122827 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-89bfc822-8de9-4796-a311-3e836b54637c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '199eb0649d2741fa8158af1736a14311'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:03.142848 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1423/2875] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 24 17:42:02 2022] DELETE /identity/v3/users/199eb0649d2741fa8158af1736a14311 => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:42:03.147607 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01df671c-0564-42d3-be7f-effff422dff1 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:03.148188 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01df671c-0564-42d3-be7f-effff422dff1 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:03.148574 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01df671c-0564-42d3-be7f-effff422dff1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:03.169539 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-01df671c-0564-42d3-be7f-effff422dff1 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:42:03.171114 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-01df671c-0564-42d3-be7f-effff422dff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:03.241458 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1453/2876] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:42:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:42:03.251327 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bfc822-8de9-4796-a311-3e836b54637c admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:03.253484 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-89bfc822-8de9-4796-a311-3e836b54637c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:03.318259 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bfc822-8de9-4796-a311-3e836b54637c admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:03.324381 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54e9700-2dd7-4200-9e45-c00fa9c35101 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e9ed4d47aef4140bfb2d3bb798cc559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:03.325041 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54e9700-2dd7-4200-9e45-c00fa9c35101 service neutron] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:03.325319 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54e9700-2dd7-4200-9e45-c00fa9c35101 service neutron] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:03.325589 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54e9700-2dd7-4200-9e45-c00fa9c35101 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:03.335617 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54e9700-2dd7-4200-9e45-c00fa9c35101 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:03.337100 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54e9700-2dd7-4200-9e45-c00fa9c35101 service neutron] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:03.347683 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1424/2877] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:42:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:42:03.888941 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01df671c-0564-42d3-be7f-effff422dff1 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:03.888941 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-01df671c-0564-42d3-be7f-effff422dff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:03.931030 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54e9700-2dd7-4200-9e45-c00fa9c35101 service neutron] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:03.933334 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-f54e9700-2dd7-4200-9e45-c00fa9c35101 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:03.971304 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54e9700-2dd7-4200-9e45-c00fa9c35101 service neutron] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:03.980336 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a92e12-b324-4edf-bcb8-afa56830028c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:03.981501 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a92e12-b324-4edf-bcb8-afa56830028c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:03.981989 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a92e12-b324-4edf-bcb8-afa56830028c admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:03.985118 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a92e12-b324-4edf-bcb8-afa56830028c admin admin] PATH_INFO: `/v3/projects/20e6abbbc746457d87ff2ba46899afdc` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:03.987240 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a92e12-b324-4edf-bcb8-afa56830028c admin admin] RBAC: Authorizing `identity:delete_project(project_id=20e6abbbc746457d87ff2ba46899afdc)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:03.989622 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a92e12-b324-4edf-bcb8-afa56830028c admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:04.010679 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:42:04.026412 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01df671c-0564-42d3-be7f-effff422dff1 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:04.032289 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a83a67-e87e-4435-b8c7-2498bc605fad service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557aca9255cb4bbb8af52e796c5c5177', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:04.033549 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a83a67-e87e-4435-b8c7-2498bc605fad service nova] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:04.033751 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a83a67-e87e-4435-b8c7-2498bc605fad service nova] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:04.034025 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a83a67-e87e-4435-b8c7-2498bc605fad service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:04.035884 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-f2a83a67-e87e-4435-b8c7-2498bc605fad service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:04.048970 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a83a67-e87e-4435-b8c7-2498bc605fad service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:04.050597 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a83a67-e87e-4435-b8c7-2498bc605fad service nova] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:04.062467 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1454/2878] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:42:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:42:04.105152 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-b8a92e12-b324-4edf-bcb8-afa56830028c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20e6abbbc746457d87ff2ba46899afdc', 'request_id': 'req-b8a92e12-b324-4edf-bcb8-afa56830028c'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:04.186017 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-b8a92e12-b324-4edf-bcb8-afa56830028c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20e6abbbc746457d87ff2ba46899afdc', 'request_id': 'req-b8a92e12-b324-4edf-bcb8-afa56830028c'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:04.196650 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-b8a92e12-b324-4edf-bcb8-afa56830028c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20e6abbbc746457d87ff2ba46899afdc', 'request_id': 'req-b8a92e12-b324-4edf-bcb8-afa56830028c'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:04.203877 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-b8a92e12-b324-4edf-bcb8-afa56830028c admin admin] The token cache is being invalidate because project 20e6abbbc746457d87ff2ba46899afdc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81732) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:42:04.204169 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-b8a92e12-b324-4edf-bcb8-afa56830028c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:04.205344 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-b8a92e12-b324-4edf-bcb8-afa56830028c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:04.210227 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1425/2879] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 24 17:42:03 2022] DELETE /identity/v3/projects/20e6abbbc746457d87ff2ba46899afdc => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:42:04.213011 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b28643c-a420-44d7-ae7e-b4492453379c None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:04.213487 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b28643c-a420-44d7-ae7e-b4492453379c None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:04.214080 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b28643c-a420-44d7-ae7e-b4492453379c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:04.236789 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-9b28643c-a420-44d7-ae7e-b4492453379c None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:42:04.238164 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-9b28643c-a420-44d7-ae7e-b4492453379c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:04.325869 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1455/2880] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:42:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:42:04.395663 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5314e615-eece-452d-a8d7-586c71b1d29f None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:04.395935 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5314e615-eece-452d-a8d7-586c71b1d29f None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:04.396267 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5314e615-eece-452d-a8d7-586c71b1d29f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:04.418215 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-5314e615-eece-452d-a8d7-586c71b1d29f None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:42:04.419466 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-5314e615-eece-452d-a8d7-586c71b1d29f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:04.428424 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1426/2881] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:42:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:42:04.432502 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b28643c-a420-44d7-ae7e-b4492453379c None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:04.442979 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b28643c-a420-44d7-ae7e-b4492453379c None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:04.447628 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0c6bff-f533-4adf-920f-ad0341f4a241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:04.448234 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0c6bff-f533-4adf-920f-ad0341f4a241 admin admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:04.448514 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0c6bff-f533-4adf-920f-ad0341f4a241 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:04.448787 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0c6bff-f533-4adf-920f-ad0341f4a241 admin admin] PATH_INFO: `/v3/domains` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:04.449462 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0c6bff-f533-4adf-920f-ad0341f4a241 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:42:04.449770 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0c6bff-f533-4adf-920f-ad0341f4a241 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:04.451395 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0c6bff-f533-4adf-920f-ad0341f4a241 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:04.461044 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=100 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:42:04.465318 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1456/2882] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 24 17:42:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 17:42:04.474016 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5314e615-eece-452d-a8d7-586c71b1d29f None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:04.483609 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5314e615-eece-452d-a8d7-586c71b1d29f None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:04.488912 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90faebcb-fd03-4482-b463-e8493474c0f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:04.489529 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90faebcb-fd03-4482-b463-e8493474c0f0 admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:04.489858 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90faebcb-fd03-4482-b463-e8493474c0f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:04.490106 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90faebcb-fd03-4482-b463-e8493474c0f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:04.490751 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90faebcb-fd03-4482-b463-e8493474c0f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:04.493309 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90faebcb-fd03-4482-b463-e8493474c0f0 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:04.582837 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1427/2883] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 24 17:42:04 2022] POST /identity/v3/projects => generated 371 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:42:04.587051 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0c6bff-f533-4adf-920f-ad0341f4a241 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:04.597784 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0c6bff-f533-4adf-920f-ad0341f4a241 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:04.656432 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c56a7a9-304a-47c9-9099-7116b62b26e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:04.657306 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c56a7a9-304a-47c9-9099-7116b62b26e5 admin admin] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:04.657612 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c56a7a9-304a-47c9-9099-7116b62b26e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:04.657884 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c56a7a9-304a-47c9-9099-7116b62b26e5 admin admin] PATH_INFO: `/v3/users` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:04.658781 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c56a7a9-304a-47c9-9099-7116b62b26e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:04.660841 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c56a7a9-304a-47c9-9099-7116b62b26e5 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:04.749618 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1457/2884] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 24 17:42:04 2022] POST /identity/v3/users => generated 286 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 17:42:04.750484 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90faebcb-fd03-4482-b463-e8493474c0f0 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:04.762112 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90faebcb-fd03-4482-b463-e8493474c0f0 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:04.767674 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b90d1df-5757-4452-9cbb-a260c66e084e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:04.768599 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b90d1df-5757-4452-9cbb-a260c66e084e admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:04.768909 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b90d1df-5757-4452-9cbb-a260c66e084e admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:04.769182 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b90d1df-5757-4452-9cbb-a260c66e084e admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:04.770222 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b90d1df-5757-4452-9cbb-a260c66e084e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:04.772217 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b90d1df-5757-4452-9cbb-a260c66e084e admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:04.784524 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1428/2885] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:42:04 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 17:42:04.788907 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c56a7a9-304a-47c9-9099-7116b62b26e5 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:04.806428 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c56a7a9-304a-47c9-9099-7116b62b26e5 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:04.811211 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6b33c6-f2e3-4aba-a5b9-b27a31c05701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:04.811893 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6b33c6-f2e3-4aba-a5b9-b27a31c05701 admin admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:04.812213 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6b33c6-f2e3-4aba-a5b9-b27a31c05701 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:04.812454 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6b33c6-f2e3-4aba-a5b9-b27a31c05701 admin admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:04.813107 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6b33c6-f2e3-4aba-a5b9-b27a31c05701 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:04.815079 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6b33c6-f2e3-4aba-a5b9-b27a31c05701 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:04.826345 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1458/2886] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:42:04 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:42:04.831427 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b90d1df-5757-4452-9cbb-a260c66e084e admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:04.841891 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b90d1df-5757-4452-9cbb-a260c66e084e admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:04.847272 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc17058-adca-411c-999c-a8d4fb6eac10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:04.847945 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc17058-adca-411c-999c-a8d4fb6eac10 admin admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:04.848226 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc17058-adca-411c-999c-a8d4fb6eac10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:04.848503 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc17058-adca-411c-999c-a8d4fb6eac10 admin admin] PATH_INFO: `/v3/projects/1bb9eefa9f8d4bc5abaec8da0597fd75/users/371fb5fc933e4ab59dca8b9c1621aa10/roles/60dc7e16aee746d5a3e8aa27d7d8525c` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:04.863948 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc17058-adca-411c-999c-a8d4fb6eac10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1bb9eefa9f8d4bc5abaec8da0597fd75, user_id=371fb5fc933e4ab59dca8b9c1621aa10, role_id=60dc7e16aee746d5a3e8aa27d7d8525c)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:04.866740 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc17058-adca-411c-999c-a8d4fb6eac10 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:04.940348 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1429/2887] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 24 17:42:04 2022] PUT /identity/v3/projects/1bb9eefa9f8d4bc5abaec8da0597fd75/users/371fb5fc933e4ab59dca8b9c1621aa10/roles/60dc7e16aee746d5a3e8aa27d7d8525c => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:42:04.950453 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643105a0-ea12-4ca0-b372-8bd74e09a883 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:04.950737 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643105a0-ea12-4ca0-b372-8bd74e09a883 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:04.956191 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643105a0-ea12-4ca0-b372-8bd74e09a883 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:04.981401 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-643105a0-ea12-4ca0-b372-8bd74e09a883 None None] MFA Rules not processed for user `371fb5fc933e4ab59dca8b9c1621aa10`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:42:04.982609 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-643105a0-ea12-4ca0-b372-8bd74e09a883 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:05.048405 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1459/2888] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:42:04 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:42:05.059738 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc17058-adca-411c-999c-a8d4fb6eac10 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:05.061443 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-4dc17058-adca-411c-999c-a8d4fb6eac10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:05.120048 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc17058-adca-411c-999c-a8d4fb6eac10 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:05.123810 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d986ef44-1aef-4692-a538-d543e10b36fa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be06246d47584b529150ceb98a5cfd95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:05.124408 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d986ef44-1aef-4692-a538-d543e10b36fa service cinder] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:05.124682 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d986ef44-1aef-4692-a538-d543e10b36fa service cinder] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:05.124959 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d986ef44-1aef-4692-a538-d543e10b36fa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:05.133983 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d986ef44-1aef-4692-a538-d543e10b36fa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:05.135436 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d986ef44-1aef-4692-a538-d543e10b36fa service cinder] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:05.144859 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1430/2889] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:42:05 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:42:05.169866 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643105a0-ea12-4ca0-b372-8bd74e09a883 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:05.234286 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643105a0-ea12-4ca0-b372-8bd74e09a883 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:05.239209 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ad455e-604f-45bc-8010-cfebaea8e089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:05.239841 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ad455e-604f-45bc-8010-cfebaea8e089 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:05.240147 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ad455e-604f-45bc-8010-cfebaea8e089 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:05.240423 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ad455e-604f-45bc-8010-cfebaea8e089 admin admin] PATH_INFO: `/v3/users/371fb5fc933e4ab59dca8b9c1621aa10` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:05.241551 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ad455e-604f-45bc-8010-cfebaea8e089 admin admin] RBAC: Authorizing `identity:delete_user(user_id=371fb5fc933e4ab59dca8b9c1621aa10)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:05.243482 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ad455e-604f-45bc-8010-cfebaea8e089 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:05.255323 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:42:05.408346 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-b0ad455e-604f-45bc-8010-cfebaea8e089 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '371fb5fc933e4ab59dca8b9c1621aa10'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:05.420496 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-b0ad455e-604f-45bc-8010-cfebaea8e089 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '371fb5fc933e4ab59dca8b9c1621aa10'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:05.497792 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-b0ad455e-604f-45bc-8010-cfebaea8e089 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '371fb5fc933e4ab59dca8b9c1621aa10'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:05.498437 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-b0ad455e-604f-45bc-8010-cfebaea8e089 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '371fb5fc933e4ab59dca8b9c1621aa10'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:05.506742 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-b0ad455e-604f-45bc-8010-cfebaea8e089 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '371fb5fc933e4ab59dca8b9c1621aa10'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:05.512743 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1460/2890] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 24 17:42:05 2022] DELETE /identity/v3/users/371fb5fc933e4ab59dca8b9c1621aa10 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:42:05.517508 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90efea70-865f-4b7a-8d36-027e158ff03e None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:05.517792 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90efea70-865f-4b7a-8d36-027e158ff03e None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:05.518062 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90efea70-865f-4b7a-8d36-027e158ff03e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:05.537423 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-90efea70-865f-4b7a-8d36-027e158ff03e None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:42:05.538625 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-90efea70-865f-4b7a-8d36-027e158ff03e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:05.583071 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:42:05.604164 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1431/2891] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:42:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:42:05.614465 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ad455e-604f-45bc-8010-cfebaea8e089 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:05.615923 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-b0ad455e-604f-45bc-8010-cfebaea8e089 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:05.680719 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ad455e-604f-45bc-8010-cfebaea8e089 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:05.684414 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fe323a-ee18-4dc0-92f0-2e571308e890 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e9ed4d47aef4140bfb2d3bb798cc559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:05.685026 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fe323a-ee18-4dc0-92f0-2e571308e890 service neutron] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:05.685307 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fe323a-ee18-4dc0-92f0-2e571308e890 service neutron] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:05.685584 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fe323a-ee18-4dc0-92f0-2e571308e890 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:05.695680 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fe323a-ee18-4dc0-92f0-2e571308e890 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:05.699358 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fe323a-ee18-4dc0-92f0-2e571308e890 service neutron] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:05.706784 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1461/2892] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:42:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:42:06.147347 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90efea70-865f-4b7a-8d36-027e158ff03e None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:06.152472 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-90efea70-865f-4b7a-8d36-027e158ff03e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:06.164047 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90efea70-865f-4b7a-8d36-027e158ff03e None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:06.170199 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b24bdd-5fa1-49c7-a321-59a3e6fd4431 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:06.171009 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b24bdd-5fa1-49c7-a321-59a3e6fd4431 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:06.171328 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b24bdd-5fa1-49c7-a321-59a3e6fd4431 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:06.171592 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b24bdd-5fa1-49c7-a321-59a3e6fd4431 admin admin] PATH_INFO: `/v3/projects/1bb9eefa9f8d4bc5abaec8da0597fd75` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:06.172971 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b24bdd-5fa1-49c7-a321-59a3e6fd4431 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1bb9eefa9f8d4bc5abaec8da0597fd75)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:06.175185 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b24bdd-5fa1-49c7-a321-59a3e6fd4431 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:06.295130 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-57b24bdd-5fa1-49c7-a321-59a3e6fd4431 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1bb9eefa9f8d4bc5abaec8da0597fd75', 'request_id': 'req-57b24bdd-5fa1-49c7-a321-59a3e6fd4431'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:06.368216 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-57b24bdd-5fa1-49c7-a321-59a3e6fd4431 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1bb9eefa9f8d4bc5abaec8da0597fd75', 'request_id': 'req-57b24bdd-5fa1-49c7-a321-59a3e6fd4431'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:06.375224 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-57b24bdd-5fa1-49c7-a321-59a3e6fd4431 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1bb9eefa9f8d4bc5abaec8da0597fd75', 'request_id': 'req-57b24bdd-5fa1-49c7-a321-59a3e6fd4431'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:06.385939 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-57b24bdd-5fa1-49c7-a321-59a3e6fd4431 admin admin] The token cache is being invalidate because project 1bb9eefa9f8d4bc5abaec8da0597fd75 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81732) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:42:06.386227 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-57b24bdd-5fa1-49c7-a321-59a3e6fd4431 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:06.386868 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-57b24bdd-5fa1-49c7-a321-59a3e6fd4431 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:06.388374 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1432/2893] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 24 17:42:06 2022] DELETE /identity/v3/projects/1bb9eefa9f8d4bc5abaec8da0597fd75 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:42:06.393656 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54ce96a-7ad1-47a6-bf80-7115fdd44195 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:06.394134 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54ce96a-7ad1-47a6-bf80-7115fdd44195 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:06.394134 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54ce96a-7ad1-47a6-bf80-7115fdd44195 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:06.414114 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-e54ce96a-7ad1-47a6-bf80-7115fdd44195 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:42:06.415340 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-e54ce96a-7ad1-47a6-bf80-7115fdd44195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:06.474290 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1462/2894] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:42:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:42:06.481888 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22283600-78b7-489e-9e07-103f39a5af45 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:06.482171 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22283600-78b7-489e-9e07-103f39a5af45 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:06.482443 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22283600-78b7-489e-9e07-103f39a5af45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:06.502425 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-22283600-78b7-489e-9e07-103f39a5af45 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:42:06.503641 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-22283600-78b7-489e-9e07-103f39a5af45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:06.511514 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1433/2895] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:42:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:42:06.516076 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54ce96a-7ad1-47a6-bf80-7115fdd44195 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:06.525724 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54ce96a-7ad1-47a6-bf80-7115fdd44195 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:06.530266 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a3c3ce-f9de-45b6-9f0e-75c8b37cd8ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:06.530855 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a3c3ce-f9de-45b6-9f0e-75c8b37cd8ac admin admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:06.531160 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a3c3ce-f9de-45b6-9f0e-75c8b37cd8ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:06.531432 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a3c3ce-f9de-45b6-9f0e-75c8b37cd8ac admin admin] PATH_INFO: `/v3/domains` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:06.532335 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a3c3ce-f9de-45b6-9f0e-75c8b37cd8ac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81731) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 17:42:06.533188 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a3c3ce-f9de-45b6-9f0e-75c8b37cd8ac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:06.534750 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a3c3ce-f9de-45b6-9f0e-75c8b37cd8ac admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:06.547809 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1463/2896] 10.222.0.18 () {58 vars in 1153 bytes} [Wed Aug 24 17:42:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 17:42:06.552160 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22283600-78b7-489e-9e07-103f39a5af45 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:06.561440 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22283600-78b7-489e-9e07-103f39a5af45 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:06.566326 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d2ff8a-888c-4ff9-9f5c-f6b77e8905b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:06.566950 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d2ff8a-888c-4ff9-9f5c-f6b77e8905b9 admin admin] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:06.567228 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d2ff8a-888c-4ff9-9f5c-f6b77e8905b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:06.567496 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d2ff8a-888c-4ff9-9f5c-f6b77e8905b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:06.568153 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d2ff8a-888c-4ff9-9f5c-f6b77e8905b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:06.570025 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d2ff8a-888c-4ff9-9f5c-f6b77e8905b9 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:06.678297 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1434/2897] 10.222.0.18 () {60 vars in 1153 bytes} [Wed Aug 24 17:42:06 2022] POST /identity/v3/projects => generated 391 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:42:06.678514 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a3c3ce-f9de-45b6-9f0e-75c8b37cd8ac admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:06.685849 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a3c3ce-f9de-45b6-9f0e-75c8b37cd8ac admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:06.698988 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:42:06.745392 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3c4bd8-6b35-422c-9103-49d640eb8dfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:06.746009 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3c4bd8-6b35-422c-9103-49d640eb8dfb admin admin] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:06.746284 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3c4bd8-6b35-422c-9103-49d640eb8dfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:06.746558 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3c4bd8-6b35-422c-9103-49d640eb8dfb admin admin] PATH_INFO: `/v3/users` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:06.747216 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3c4bd8-6b35-422c-9103-49d640eb8dfb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:06.749293 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3c4bd8-6b35-422c-9103-49d640eb8dfb admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:06.847212 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1464/2898] 10.222.0.18 () {60 vars in 1144 bytes} [Wed Aug 24 17:42:06 2022] POST /identity/v3/users => generated 296 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 17:42:06.851306 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d2ff8a-888c-4ff9-9f5c-f6b77e8905b9 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:06.861809 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d2ff8a-888c-4ff9-9f5c-f6b77e8905b9 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:06.866710 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b971f2c-8433-44a7-9223-39ad4f846d0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:06.867412 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b971f2c-8433-44a7-9223-39ad4f846d0c admin admin] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:06.867689 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b971f2c-8433-44a7-9223-39ad4f846d0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:06.867980 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b971f2c-8433-44a7-9223-39ad4f846d0c admin admin] PATH_INFO: `/v3/roles` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:06.868609 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b971f2c-8433-44a7-9223-39ad4f846d0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:06.870431 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b971f2c-8433-44a7-9223-39ad4f846d0c admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:06.880468 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1435/2899] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:42:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:42:06.884473 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3c4bd8-6b35-422c-9103-49d640eb8dfb admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:06.895072 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3c4bd8-6b35-422c-9103-49d640eb8dfb admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:06.899668 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1548bc8-923b-40af-b4c1-f5c4dac3b79e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:06.900267 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1548bc8-923b-40af-b4c1-f5c4dac3b79e admin admin] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:06.900544 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1548bc8-923b-40af-b4c1-f5c4dac3b79e admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:06.900818 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1548bc8-923b-40af-b4c1-f5c4dac3b79e admin admin] PATH_INFO: `/v3/roles` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:06.901453 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1548bc8-923b-40af-b4c1-f5c4dac3b79e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:06.903095 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1548bc8-923b-40af-b4c1-f5c4dac3b79e admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:06.914490 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1465/2900] 10.222.0.18 () {58 vars in 1122 bytes} [Wed Aug 24 17:42:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 17:42:06.918881 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b971f2c-8433-44a7-9223-39ad4f846d0c admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:06.928761 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b971f2c-8433-44a7-9223-39ad4f846d0c admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:06.933231 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40201871-4cdf-4a72-b085-0afdafe0fc0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:06.933865 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40201871-4cdf-4a72-b085-0afdafe0fc0b admin admin] REQUEST_METHOD: `PUT` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:06.934140 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40201871-4cdf-4a72-b085-0afdafe0fc0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:06.934409 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40201871-4cdf-4a72-b085-0afdafe0fc0b admin admin] PATH_INFO: `/v3/projects/f29a166d89a54cbaab7635dd81d3695a/users/c80edb5d9eae441c944ce09cefec0b86/roles/60dc7e16aee746d5a3e8aa27d7d8525c` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:06.949732 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40201871-4cdf-4a72-b085-0afdafe0fc0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f29a166d89a54cbaab7635dd81d3695a, user_id=c80edb5d9eae441c944ce09cefec0b86, role_id=60dc7e16aee746d5a3e8aa27d7d8525c)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:06.952413 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40201871-4cdf-4a72-b085-0afdafe0fc0b admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:07.025489 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1436/2901] 10.222.0.18 () {60 vars in 1483 bytes} [Wed Aug 24 17:42:06 2022] PUT /identity/v3/projects/f29a166d89a54cbaab7635dd81d3695a/users/c80edb5d9eae441c944ce09cefec0b86/roles/60dc7e16aee746d5a3e8aa27d7d8525c => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:42:07.037839 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dec0894-5d04-4edd-ad22-8cc3ede00ee9 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:07.038127 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dec0894-5d04-4edd-ad22-8cc3ede00ee9 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:07.038406 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dec0894-5d04-4edd-ad22-8cc3ede00ee9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:07.068288 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-3dec0894-5d04-4edd-ad22-8cc3ede00ee9 None None] MFA Rules not processed for user `c80edb5d9eae441c944ce09cefec0b86`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:42:07.069482 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-3dec0894-5d04-4edd-ad22-8cc3ede00ee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:07.132297 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1466/2902] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:42:07 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:42:07.143070 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40201871-4cdf-4a72-b085-0afdafe0fc0b admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:07.149066 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-40201871-4cdf-4a72-b085-0afdafe0fc0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:07.201629 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40201871-4cdf-4a72-b085-0afdafe0fc0b admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:07.205283 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d0f4fd-7ce5-4fda-9dcf-bb1e0f2a074a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be06246d47584b529150ceb98a5cfd95', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:07.205887 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d0f4fd-7ce5-4fda-9dcf-bb1e0f2a074a service cinder] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:07.206170 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d0f4fd-7ce5-4fda-9dcf-bb1e0f2a074a service cinder] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:07.206477 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d0f4fd-7ce5-4fda-9dcf-bb1e0f2a074a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:07.214864 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d0f4fd-7ce5-4fda-9dcf-bb1e0f2a074a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:07.216318 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d0f4fd-7ce5-4fda-9dcf-bb1e0f2a074a service cinder] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:07.226181 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1437/2903] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:42:07 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:42:14.745277 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dec0894-5d04-4edd-ad22-8cc3ede00ee9 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:14.746719 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-3dec0894-5d04-4edd-ad22-8cc3ede00ee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:14.813741 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dec0894-5d04-4edd-ad22-8cc3ede00ee9 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:14.818676 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8fa0d3-933f-4c68-9eb8-9f7c6ce1c99f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e9ed4d47aef4140bfb2d3bb798cc559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:14.819336 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8fa0d3-933f-4c68-9eb8-9f7c6ce1c99f service neutron] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:14.819613 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8fa0d3-933f-4c68-9eb8-9f7c6ce1c99f service neutron] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:14.819885 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8fa0d3-933f-4c68-9eb8-9f7c6ce1c99f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:14.821685 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-cf8fa0d3-933f-4c68-9eb8-9f7c6ce1c99f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:14.831802 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8fa0d3-933f-4c68-9eb8-9f7c6ce1c99f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:14.833228 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8fa0d3-933f-4c68-9eb8-9f7c6ce1c99f service neutron] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:14.841610 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1467/2904] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:42:14 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 17:42:17.221575 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=87,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:42:24.837838 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=84,keystone:INSERT=3 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:42:29.514042 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d0f4fd-7ce5-4fda-9dcf-bb1e0f2a074a service cinder] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:29.515719 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-15d0f4fd-7ce5-4fda-9dcf-bb1e0f2a074a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:29.530201 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d0f4fd-7ce5-4fda-9dcf-bb1e0f2a074a service cinder] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:29.534214 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f0aa0f-e521-47b6-a7ea-ca944aad3328 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e9ed4d47aef4140bfb2d3bb798cc559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:29.535393 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f0aa0f-e521-47b6-a7ea-ca944aad3328 service neutron] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:29.535674 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f0aa0f-e521-47b6-a7ea-ca944aad3328 service neutron] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:29.535946 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f0aa0f-e521-47b6-a7ea-ca944aad3328 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:29.537698 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-23f0aa0f-e521-47b6-a7ea-ca944aad3328 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:29.622231 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f0aa0f-e521-47b6-a7ea-ca944aad3328 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:29.623701 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f0aa0f-e521-47b6-a7ea-ca944aad3328 service neutron] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:29.633343 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1438/2905] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:42:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:42:32.552042 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8fa0d3-933f-4c68-9eb8-9f7c6ce1c99f service neutron] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:32.579848 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8fa0d3-933f-4c68-9eb8-9f7c6ce1c99f service neutron] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:32.585658 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ca9498-3964-41f8-a366-3ed943ac89e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:32.586590 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ca9498-3964-41f8-a366-3ed943ac89e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:32.587035 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ca9498-3964-41f8-a366-3ed943ac89e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:32.587378 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ca9498-3964-41f8-a366-3ed943ac89e1 admin admin] PATH_INFO: `/v3/users/c80edb5d9eae441c944ce09cefec0b86` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:32.588806 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ca9498-3964-41f8-a366-3ed943ac89e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c80edb5d9eae441c944ce09cefec0b86)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:32.591388 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ca9498-3964-41f8-a366-3ed943ac89e1 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:32.772902 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-07ca9498-3964-41f8-a366-3ed943ac89e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c80edb5d9eae441c944ce09cefec0b86'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:32.793075 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-07ca9498-3964-41f8-a366-3ed943ac89e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c80edb5d9eae441c944ce09cefec0b86'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:32.883049 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-07ca9498-3964-41f8-a366-3ed943ac89e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c80edb5d9eae441c944ce09cefec0b86'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:32.883809 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-07ca9498-3964-41f8-a366-3ed943ac89e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c80edb5d9eae441c944ce09cefec0b86'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:32.895868 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-07ca9498-3964-41f8-a366-3ed943ac89e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c80edb5d9eae441c944ce09cefec0b86'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:32.902438 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1468/2906] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 24 17:42:32 2022] DELETE /identity/v3/users/c80edb5d9eae441c944ce09cefec0b86 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:42:32.907404 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377db7f4-338d-4601-a80e-db22683c0824 None None] REQUEST_METHOD: `POST` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:32.907985 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377db7f4-338d-4601-a80e-db22683c0824 None None] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:32.908187 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377db7f4-338d-4601-a80e-db22683c0824 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:32.933213 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.auth.core [None req-377db7f4-338d-4601-a80e-db22683c0824 None None] MFA Rules not processed for user `20b73f4d8b424258a870e87b04c9a532`. Rule list: `[]` (Enabled: `True`). {{(pid=81732) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:42:32.934565 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-377db7f4-338d-4601-a80e-db22683c0824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:33.010698 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1439/2907] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:42:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:42:33.018047 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ca9498-3964-41f8-a366-3ed943ac89e1 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:33.020624 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-07ca9498-3964-41f8-a366-3ed943ac89e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:33.121092 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ca9498-3964-41f8-a366-3ed943ac89e1 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:33.125796 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b12319c-2040-4750-a2fb-d761c9a229b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e9ed4d47aef4140bfb2d3bb798cc559', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:33.126433 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b12319c-2040-4750-a2fb-d761c9a229b3 service neutron] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:33.126717 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b12319c-2040-4750-a2fb-d761c9a229b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:33.127019 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b12319c-2040-4750-a2fb-d761c9a229b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:33.138677 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b12319c-2040-4750-a2fb-d761c9a229b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:33.140210 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b12319c-2040-4750-a2fb-d761c9a229b3 service neutron] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:33.152089 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1469/2908] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:42:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:42:33.758480 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377db7f4-338d-4601-a80e-db22683c0824 None None] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:33.760352 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-377db7f4-338d-4601-a80e-db22683c0824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:33.783194 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377db7f4-338d-4601-a80e-db22683c0824 None None] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:33.789936 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d747484-ab77-448a-b294-ecca00759080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:33.790810 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d747484-ab77-448a-b294-ecca00759080 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:33.791190 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d747484-ab77-448a-b294-ecca00759080 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:33.791466 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d747484-ab77-448a-b294-ecca00759080 admin admin] PATH_INFO: `/v3/projects/f29a166d89a54cbaab7635dd81d3695a` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:33.792955 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d747484-ab77-448a-b294-ecca00759080 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f29a166d89a54cbaab7635dd81d3695a)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:33.795232 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d747484-ab77-448a-b294-ecca00759080 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:33.930368 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-2d747484-ab77-448a-b294-ecca00759080 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f29a166d89a54cbaab7635dd81d3695a', 'request_id': 'req-2d747484-ab77-448a-b294-ecca00759080'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:34.016997 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-2d747484-ab77-448a-b294-ecca00759080 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f29a166d89a54cbaab7635dd81d3695a', 'request_id': 'req-2d747484-ab77-448a-b294-ecca00759080'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:34.024954 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-2d747484-ab77-448a-b294-ecca00759080 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f29a166d89a54cbaab7635dd81d3695a', 'request_id': 'req-2d747484-ab77-448a-b294-ecca00759080'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:34.038325 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-2d747484-ab77-448a-b294-ecca00759080 admin admin] The token cache is being invalidate because project f29a166d89a54cbaab7635dd81d3695a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81732) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:42:34.038625 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-2d747484-ab77-448a-b294-ecca00759080 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:34.039324 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-2d747484-ab77-448a-b294-ecca00759080 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:34.040871 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1440/2909] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 24 17:42:33 2022] DELETE /identity/v3/projects/f29a166d89a54cbaab7635dd81d3695a => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:42:37.295616 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b12319c-2040-4750-a2fb-d761c9a229b3 service neutron] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:37.297174 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-4b12319c-2040-4750-a2fb-d761c9a229b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:37.313775 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b12319c-2040-4750-a2fb-d761c9a229b3 service neutron] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:37.371691 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bfe2ba-a467-442e-aa8f-0af4b15d31fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:37.372314 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bfe2ba-a467-442e-aa8f-0af4b15d31fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:37.372590 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bfe2ba-a467-442e-aa8f-0af4b15d31fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:37.372861 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bfe2ba-a467-442e-aa8f-0af4b15d31fd admin admin] PATH_INFO: `/v3/users/4df56db015514caa8aa92ffcfe227914` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:37.374000 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bfe2ba-a467-442e-aa8f-0af4b15d31fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=4df56db015514caa8aa92ffcfe227914)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:37.375980 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bfe2ba-a467-442e-aa8f-0af4b15d31fd admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:37.391335 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:42:37.579198 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-04bfe2ba-a467-442e-aa8f-0af4b15d31fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4df56db015514caa8aa92ffcfe227914'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:37.591346 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-04bfe2ba-a467-442e-aa8f-0af4b15d31fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4df56db015514caa8aa92ffcfe227914'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:37.676771 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-04bfe2ba-a467-442e-aa8f-0af4b15d31fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4df56db015514caa8aa92ffcfe227914'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:37.677353 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-04bfe2ba-a467-442e-aa8f-0af4b15d31fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4df56db015514caa8aa92ffcfe227914'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:37.685417 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-04bfe2ba-a467-442e-aa8f-0af4b15d31fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4df56db015514caa8aa92ffcfe227914'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:37.690946 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1470/2910] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 24 17:42:37 2022] DELETE /identity/v3/users/4df56db015514caa8aa92ffcfe227914 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:42:37.978603 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d747484-ab77-448a-b294-ecca00759080 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:37.980076 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-2d747484-ab77-448a-b294-ecca00759080 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:37.993677 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d747484-ab77-448a-b294-ecca00759080 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:38.007707 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:42:38.056274 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9961f41e-7855-4671-b47c-010a80290216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:38.056932 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9961f41e-7855-4671-b47c-010a80290216 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:38.057211 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9961f41e-7855-4671-b47c-010a80290216 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:38.057482 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9961f41e-7855-4671-b47c-010a80290216 admin admin] PATH_INFO: `/v3/projects/2a66fdbac83c47b59c88adbe6b82ff8a` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:38.058765 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9961f41e-7855-4671-b47c-010a80290216 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a66fdbac83c47b59c88adbe6b82ff8a)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:38.060749 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9961f41e-7855-4671-b47c-010a80290216 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:38.174671 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-9961f41e-7855-4671-b47c-010a80290216 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a66fdbac83c47b59c88adbe6b82ff8a', 'request_id': 'req-9961f41e-7855-4671-b47c-010a80290216'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:38.259134 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-9961f41e-7855-4671-b47c-010a80290216 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a66fdbac83c47b59c88adbe6b82ff8a', 'request_id': 'req-9961f41e-7855-4671-b47c-010a80290216'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:38.265966 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-9961f41e-7855-4671-b47c-010a80290216 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a66fdbac83c47b59c88adbe6b82ff8a', 'request_id': 'req-9961f41e-7855-4671-b47c-010a80290216'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:38.277677 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-9961f41e-7855-4671-b47c-010a80290216 admin admin] The token cache is being invalidate because project 2a66fdbac83c47b59c88adbe6b82ff8a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81732) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:42:38.277962 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-9961f41e-7855-4671-b47c-010a80290216 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:38.278601 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-9961f41e-7855-4671-b47c-010a80290216 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:38.280096 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1441/2911] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 24 17:42:37 2022] DELETE /identity/v3/projects/2a66fdbac83c47b59c88adbe6b82ff8a => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:42:38.284355 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bfe2ba-a467-442e-aa8f-0af4b15d31fd admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:38.287319 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-04bfe2ba-a467-442e-aa8f-0af4b15d31fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:38.304907 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bfe2ba-a467-442e-aa8f-0af4b15d31fd admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:38.363348 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce35c9b7-5696-41e6-ac2f-63c31b26def2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:38.364029 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce35c9b7-5696-41e6-ac2f-63c31b26def2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:38.364308 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce35c9b7-5696-41e6-ac2f-63c31b26def2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:38.364578 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce35c9b7-5696-41e6-ac2f-63c31b26def2 admin admin] PATH_INFO: `/v3/users/1c739fda2d7e4281aebc311b05a64e08` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:38.365683 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce35c9b7-5696-41e6-ac2f-63c31b26def2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c739fda2d7e4281aebc311b05a64e08)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:38.367787 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce35c9b7-5696-41e6-ac2f-63c31b26def2 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:38.545486 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-ce35c9b7-5696-41e6-ac2f-63c31b26def2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c739fda2d7e4281aebc311b05a64e08'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:38.557976 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-ce35c9b7-5696-41e6-ac2f-63c31b26def2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c739fda2d7e4281aebc311b05a64e08'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:38.636675 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-ce35c9b7-5696-41e6-ac2f-63c31b26def2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c739fda2d7e4281aebc311b05a64e08'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:38.637328 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-ce35c9b7-5696-41e6-ac2f-63c31b26def2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c739fda2d7e4281aebc311b05a64e08'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:38.645828 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-ce35c9b7-5696-41e6-ac2f-63c31b26def2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c739fda2d7e4281aebc311b05a64e08'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:38.652138 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1471/2912] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 24 17:42:38 2022] DELETE /identity/v3/users/1c739fda2d7e4281aebc311b05a64e08 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:42:38.912872 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9961f41e-7855-4671-b47c-010a80290216 admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:38.920301 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-9961f41e-7855-4671-b47c-010a80290216 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:38.928074 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9961f41e-7855-4671-b47c-010a80290216 admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:38.986706 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a68612-3aab-4e1e-a2d3-95a183ab32cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:38.987603 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a68612-3aab-4e1e-a2d3-95a183ab32cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:38.988493 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a68612-3aab-4e1e-a2d3-95a183ab32cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:38.988835 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a68612-3aab-4e1e-a2d3-95a183ab32cf admin admin] PATH_INFO: `/v3/projects/6942c8e041c94e1a83d1688851bd0fe4` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:38.990018 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a68612-3aab-4e1e-a2d3-95a183ab32cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=6942c8e041c94e1a83d1688851bd0fe4)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:38.991993 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a68612-3aab-4e1e-a2d3-95a183ab32cf admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:39.122986 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-03a68612-3aab-4e1e-a2d3-95a183ab32cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6942c8e041c94e1a83d1688851bd0fe4', 'request_id': 'req-03a68612-3aab-4e1e-a2d3-95a183ab32cf'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:39.124683 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:42:39.207988 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-03a68612-3aab-4e1e-a2d3-95a183ab32cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6942c8e041c94e1a83d1688851bd0fe4', 'request_id': 'req-03a68612-3aab-4e1e-a2d3-95a183ab32cf'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:39.214747 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-03a68612-3aab-4e1e-a2d3-95a183ab32cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6942c8e041c94e1a83d1688851bd0fe4', 'request_id': 'req-03a68612-3aab-4e1e-a2d3-95a183ab32cf'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:39.225798 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-03a68612-3aab-4e1e-a2d3-95a183ab32cf admin admin] The token cache is being invalidate because project 6942c8e041c94e1a83d1688851bd0fe4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81732) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:42:39.226083 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-03a68612-3aab-4e1e-a2d3-95a183ab32cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:39.229065 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-03a68612-3aab-4e1e-a2d3-95a183ab32cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:42:39.229065 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1442/2913] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 24 17:42:38 2022] DELETE /identity/v3/projects/6942c8e041c94e1a83d1688851bd0fe4 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:42:43.574314 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c75144-29ab-4243-926a-2cdc36d9f926 None None] REQUEST_METHOD: `POST` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:43.574840 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c75144-29ab-4243-926a-2cdc36d9f926 None None] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:43.574983 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c75144-29ab-4243-926a-2cdc36d9f926 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:43.595462 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.auth.core [None req-f6c75144-29ab-4243-926a-2cdc36d9f926 None None] MFA Rules not processed for user `b4076a4c51a040a094aaae066772f9d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81731) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 17:42:43.596714 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-f6c75144-29ab-4243-926a-2cdc36d9f926 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:43.605318 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:DELETE=19,keystone:SELECT=79,keystone:INSERT=2 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:42:43.652748 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1472/2914] 10.222.0.18 () {58 vars in 958 bytes} [Wed Aug 24 17:42:43 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 17:42:44.295431 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a68612-3aab-4e1e-a2d3-95a183ab32cf admin admin] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:44.296902 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-03a68612-3aab-4e1e-a2d3-95a183ab32cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:44.371093 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a68612-3aab-4e1e-a2d3-95a183ab32cf admin admin] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:44.376053 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c074cb9-1361-47ea-a206-3bcc74f3ffc8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03fc77c6107e430f90f7b6f7cfd4ec02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:44.376661 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c074cb9-1361-47ea-a206-3bcc74f3ffc8 service placement] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:44.376938 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c074cb9-1361-47ea-a206-3bcc74f3ffc8 service placement] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:44.377208 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c074cb9-1361-47ea-a206-3bcc74f3ffc8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:44.379052 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-7c074cb9-1361-47ea-a206-3bcc74f3ffc8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:44.390175 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c074cb9-1361-47ea-a206-3bcc74f3ffc8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:44.391776 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c074cb9-1361-47ea-a206-3bcc74f3ffc8 service placement] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:44.401415 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1443/2915] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:42:44 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:42:53.566340 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c75144-29ab-4243-926a-2cdc36d9f926 None None] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:42:53.567880 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-f6c75144-29ab-4243-926a-2cdc36d9f926 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:53.646318 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c75144-29ab-4243-926a-2cdc36d9f926 None None] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:42:53.652328 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af675e83-d32b-4844-99ae-14741421ae70 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '557aca9255cb4bbb8af52e796c5c5177', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:42:53.652945 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af675e83-d32b-4844-99ae-14741421ae70 service nova] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:42:53.653228 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af675e83-d32b-4844-99ae-14741421ae70 service nova] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:42:53.653522 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af675e83-d32b-4844-99ae-14741421ae70 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:42:53.655312 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-af675e83-d32b-4844-99ae-14741421ae70 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:42:53.668870 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af675e83-d32b-4844-99ae-14741421ae70 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:42:53.673232 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af675e83-d32b-4844-99ae-14741421ae70 service nova] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:42:53.681498 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1473/2916] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:42:53 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:42:54.397301 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:INSERT=1,keystone:DELETE=3,keystone:SELECT=38 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:43:03.677362 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=53 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:43:55.982206 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c074cb9-1361-47ea-a206-3bcc74f3ffc8 service placement] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:43:56.013776 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c074cb9-1361-47ea-a206-3bcc74f3ffc8 service placement] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:43:56.019654 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f452b93a-25e6-404b-9ec6-a06f01f5188e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03fc77c6107e430f90f7b6f7cfd4ec02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:43:56.020268 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f452b93a-25e6-404b-9ec6-a06f01f5188e service placement] REQUEST_METHOD: `GET` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:43:56.020547 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f452b93a-25e6-404b-9ec6-a06f01f5188e service placement] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:43:56.020820 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f452b93a-25e6-404b-9ec6-a06f01f5188e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:43:56.022546 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.fernet_utils [None req-f452b93a-25e6-404b-9ec6-a06f01f5188e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81732) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:43:56.047031 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f452b93a-25e6-404b-9ec6-a06f01f5188e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:43:56.048613 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f452b93a-25e6-404b-9ec6-a06f01f5188e service placement] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:43:56.058537 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1444/2917] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:43:55 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:43:56.093802 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af675e83-d32b-4844-99ae-14741421ae70 service nova] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:43:56.095363 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-af675e83-d32b-4844-99ae-14741421ae70 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:43:56.187339 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af675e83-d32b-4844-99ae-14741421ae70 service nova] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:43:56.191922 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2209c1-2b9b-439b-9208-8c02835fef90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:43:56.192545 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2209c1-2b9b-439b-9208-8c02835fef90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:43:56.192825 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2209c1-2b9b-439b-9208-8c02835fef90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:43:56.193094 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2209c1-2b9b-439b-9208-8c02835fef90 admin admin] PATH_INFO: `/v3/users/b4076a4c51a040a094aaae066772f9d2` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:43:56.194207 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2209c1-2b9b-439b-9208-8c02835fef90 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b4076a4c51a040a094aaae066772f9d2)` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:43:56.196138 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2209c1-2b9b-439b-9208-8c02835fef90 admin admin] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:43:56.370843 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-4e2209c1-2b9b-439b-9208-8c02835fef90 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b4076a4c51a040a094aaae066772f9d2'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:43:56.383462 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-4e2209c1-2b9b-439b-9208-8c02835fef90 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b4076a4c51a040a094aaae066772f9d2'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:43:56.443599 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-4e2209c1-2b9b-439b-9208-8c02835fef90 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b4076a4c51a040a094aaae066772f9d2'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:43:56.444219 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-4e2209c1-2b9b-439b-9208-8c02835fef90 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b4076a4c51a040a094aaae066772f9d2'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:43:56.451510 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.notifications [None req-4e2209c1-2b9b-439b-9208-8c02835fef90 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b4076a4c51a040a094aaae066772f9d2'} {{(pid=81731) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:43:56.457823 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1474/2918] 10.222.0.18 () {58 vars in 1224 bytes} [Wed Aug 24 17:43:56 2022] DELETE /identity/v3/users/b4076a4c51a040a094aaae066772f9d2 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 17:43:56.701970 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f452b93a-25e6-404b-9ec6-a06f01f5188e service placement] Authenticating user token {{(pid=81732) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:43:56.714368 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f452b93a-25e6-404b-9ec6-a06f01f5188e service placement] Validating token access rules against request {{(pid=81732) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:43:56.770508 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94cbb15-194f-4c2b-8a79-4708562b3428 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20b73f4d8b424258a870e87b04c9a532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68a3eec6caa5424ea6bb39d5d6e46c2f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81732) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:43:56.771174 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94cbb15-194f-4c2b-8a79-4708562b3428 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:43:56.771456 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94cbb15-194f-4c2b-8a79-4708562b3428 admin admin] SCRIPT_NAME: `/identity` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:43:56.771729 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94cbb15-194f-4c2b-8a79-4708562b3428 admin admin] PATH_INFO: `/v3/projects/7ebcc5baa03b430bb50f4110c5a79df9` {{(pid=81732) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:43:56.772850 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94cbb15-194f-4c2b-8a79-4708562b3428 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ebcc5baa03b430bb50f4110c5a79df9)` {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:43:56.774741 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94cbb15-194f-4c2b-8a79-4708562b3428 admin admin] RBAC: Authorization granted {{(pid=81732) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:43:56.875113 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-f94cbb15-194f-4c2b-8a79-4708562b3428 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ebcc5baa03b430bb50f4110c5a79df9', 'request_id': 'req-f94cbb15-194f-4c2b-8a79-4708562b3428'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:43:56.938027 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-f94cbb15-194f-4c2b-8a79-4708562b3428 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ebcc5baa03b430bb50f4110c5a79df9', 'request_id': 'req-f94cbb15-194f-4c2b-8a79-4708562b3428'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:43:56.944620 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-f94cbb15-194f-4c2b-8a79-4708562b3428 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ebcc5baa03b430bb50f4110c5a79df9', 'request_id': 'req-f94cbb15-194f-4c2b-8a79-4708562b3428'} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:43:56.955210 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-f94cbb15-194f-4c2b-8a79-4708562b3428 admin admin] The token cache is being invalidate because project 7ebcc5baa03b430bb50f4110c5a79df9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81732) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 17:43:56.955501 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-f94cbb15-194f-4c2b-8a79-4708562b3428 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:43:56.956137 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG keystone.notifications [None req-f94cbb15-194f-4c2b-8a79-4708562b3428 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81732) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 17:43:56.957695 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: [pid: 81732|app: 0|req: 1445/2919] 10.222.0.18 () {58 vars in 1233 bytes} [Wed Aug 24 17:43:56 2022] DELETE /identity/v3/projects/7ebcc5baa03b430bb50f4110c5a79df9 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 17:44:06.454083 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=65,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:44:06.952160 dv-cinder-851640-4-iscsi devstack@keystone.service[81732]: DEBUG dbcounter [-] [81732] Writing DB stats keystone:SELECT=69,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81732) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 17:45:11.791990 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2209c1-2b9b-439b-9208-8c02835fef90 admin admin] Authenticating user token {{(pid=81731) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 17:45:11.793367 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-4e2209c1-2b9b-439b-9208-8c02835fef90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:45:11.906016 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2209c1-2b9b-439b-9208-8c02835fef90 admin admin] Validating token access rules against request {{(pid=81731) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 17:45:11.963550 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35acb70-3cf6-4385-9fd7-5c2ad6b1996b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03fc77c6107e430f90f7b6f7cfd4ec02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '46ab158bc3644c888557781b0630b0b8', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81731) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 17:45:11.964153 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35acb70-3cf6-4385-9fd7-5c2ad6b1996b service placement] REQUEST_METHOD: `GET` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 17:45:11.964429 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35acb70-3cf6-4385-9fd7-5c2ad6b1996b service placement] SCRIPT_NAME: `/identity` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 17:45:11.964695 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35acb70-3cf6-4385-9fd7-5c2ad6b1996b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81731) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 17:45:11.966397 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.fernet_utils [None req-f35acb70-3cf6-4385-9fd7-5c2ad6b1996b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81731) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 17:45:11.978202 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35acb70-3cf6-4385-9fd7-5c2ad6b1996b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 17:45:11.979822 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35acb70-3cf6-4385-9fd7-5c2ad6b1996b service placement] RBAC: Authorization granted {{(pid=81731) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 17:45:11.990708 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: [pid: 81731|app: 0|req: 1475/2920] 10.222.0.18 () {60 vars in 1365 bytes} [Wed Aug 24 17:45:11 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 17:45:21.985640 dv-cinder-851640-4-iscsi devstack@keystone.service[81731]: DEBUG dbcounter [-] [81731] Writing DB stats keystone:SELECT=77 {{(pid=81731) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}